Scalable, high-performance data validation in hydrometeorological applications

In the era of big data, the increasing number of sensors and their data necessitate new approaches to meet the requirements. Especially, data validation is a crucial part since the data quality will eventually affect the whole following processes using data. In hydrometeorology, recent unusual weather events and climate change make the value of the qualified data deserves more attention. But, in the current solution, validation tasks with a large volume of domain data are not manageable due to the lack of scalability. Therefore a new framework is highly motivated to provide scalability with high performance. Additionally, since it is subject to further development, it needs flexibility toward future changes. Here in this paper, we present a process of modeling, designing, and implementing a new framework, which is scalable and flexible to validate a large number of domain data. Also, we will discuss various challenges and corresponding future works based on an analysis of the proposed framework. Such a discussion will make this thesis a good reference for future development.