受外界环境和仪器设备等的影响,实时水位观测数据流噪声和数据异常问题突出,严重制约了实时应用效能。针对已有数据清洗方法适应性差,难以根据动态观测数据的变化特征进行动态调整问题,本文提出了一种水文变化语义约束的实时水位观测数据流在线滤波方法:在实时水位观测数据变化特征与水文时空过程动态演变规律之间建立高层语义映射,实现水文变化语义知识约束下的卡尔曼模型参数自适应调整,从而突破传统滤波方法的瓶颈。采用多种降雨情景下的实时水位观测数据进行了试验,证明了该方法结果质量的可靠性。
Irregular environmental changes and occasional instrument malfunctions have made noises and exceptions in observational data prominence. Therefore, before processing real-time water level data online, data cleaning is urgently needed to ensure data quality. Since traditional data filtering methods didn't take the data change pattern into consideration, these methods have encountered some severe problems, including the poor adaptability of filter model, the low estimation precision and prohibitively high calculation cost. To overcome these shortcomings, this paper presents a hydrological change semantics constrained online Kalman filtering method: creating dynamic semantic mapping between real-time data changing pattern and the rules of spatial-temporal hydrological process evolution; implementing the change semantic constrained Kalman filtering method to support the adaptive parameter optimization. Observational water level data streams of different precipitation scenarios are selected for testing. Experimental results prove that by means of this method, more accurate and reliable water level information can be available.
[1] DING Yulin, FAN Yida, DU Zhiqiang, et al. An Integrated Geospatial Information Service System for Disaster Management in China[J]. International Journal of Digital Earth,2015, 8(11): 918-945.
[2] DING Yulin, ZHU Qing, LIN Hui. An Integrated Virtual Geographic Environmental Simulation Framework: A Case Study of Flood Disaster Simulation[J]. Geo-spatial Information Science,2014, 17(4): 190-200.
[3] DING Yulin, DU Zhiqiang, ZHU Qing, et al. Adaptive Water Level Correction Algorithm for Flooding Analysis[J]. Acta Geodaetica et Cartographica Sinica,2013, 42(4): 546-553. (丁雨淋, 杜志强, 朱庆, 等. 洪水淹没分析中的自适应逐点水位修正算法[J]. 测绘学报,2013, 42(4): 546-553.)
[4] SUI D, GOODCHILD M. The Convergence of GIS and Social Media: Challenges for GIScience[J]. International Journal of Geographical Information Science,2011, 25(11): 1737-1748.
[5] O'REILLY C, GLUHAK A, IMRAN M A, et al. Anomaly Detection in Wireless Sensor Networks in a Non-stationary Environment[J]. IEEE Communications Surveys & Tutorials, 2014, 16(3): 1413-1432.
[6] SMITH D, TIMMS G, De SOUZA P, et al. A Bayesian Framework for the Automated Online Assessment of Sensor Data Quality[J]. Sensors,2012, 12(7): 9476-9501.
[7] RASSAM M A, MAAROF M A, ZAINAL A. Adaptive and Online Data Anomaly Detection for Wireless Sensor Systems[J]. Knowledge-based Systems,2014, 60: 44-57.
[8] CHANDOLA V, BANERJEE A, KUMAR V. Anomaly Detection: A Survey[J]. ACM Computing Surveys,2009, 41(3): 15.
[9] HILL D J. Automated Bayesian Quality Control of Streaming Rain Gauge Data[J]. Environmental Modelling & Software,2013, 40: 289-301.
[10] PATCHA A, PARK J M. An Overview of Anomaly Detection Techniques: Existing Solutions and Latest Technological Trends[J]. Computer Networks,2007, 51(12): 3448-3470.
[11] LIU Hancong, SHAH S, JIANG Wei. On-line Outlier Detection and Data Cleaning[J]. Computers &Chemical Engineering,2004, 28(9): 1635-1647.
[12] INGLEBY B, HUDDLESTON M. Quality Control of Ocean Temperature and Salinity Profiles-historical and Real-time Data[J]. Journal of Marine Systems,2007, 65(1-4): 158-175.
[13] GELMAN A, CARLIN J B, STERN H S, et al. Bayesian Data Analysis[M]. 3rd ed. Boca Raton: Chapman and Hall/CRC press, 2013.
[14] TROYANSKAYA O G, DOLINSKI K, OWEN A B, et al. A Bayesian Framework for Combining Heterogeneous Data Sources for Gene Function Prediction (In Saccharomyces Cerevisiae)[C]//Proceedings of the National Academy of Sciences of the United States of America,2003, 100(14): 8348-8353.
[15] KANO M,NAKAGAWA Y. Data-based Process Monitoring, Process Control, and Quality Improvement: Recent Developments and Applications in Steel Industry[J]. Computers & Chemical Engineering,2008, 32(1-2): 12-24.
[16] LIU Fanming,QIAN Dong, GUO Jing. Terrain Estimation Algorithm Based on Kalman Filter and Its Simulation Research[J]. Acta Geodaetica et Cartographica Sinica,2011, 40(1): 45-51. (刘繁明, 钱东, 郭静. 基于卡尔曼滤波的地形反演方法及其仿真研究[J]. 测绘学报,2011, 40(1): 45-51.)
[17] LIU Guolin, HAO Huadong, YAN Man, et al. Phase Unwrapping Algorithm by Using Kalman Filter Based on Topographic Factors[J]. Acta Geodaetica et Cartographica Sinica,2011, 40(3): 283-288. (刘国林, 郝华东, 闫满, 等. 顾及地形因素的卡尔曼滤波相位解缠算法[J]. 测绘学报,2011, 40(3): 283-288.)
[18] LI Lihua, PENG Junhuan. Multiple Kalman Filters Model with Shaping Filter GPS Real-time Deformation Analysis[J]. Transactions of Nonferrous Metals Society of China,2014, 24(11): 3674-3681.
[19] YAN Xiaozhen, LUO Qinghua. Dynamic Sensor Data Stream Estimation Method Based on Kalman Filtering[J]. Chinese Journal of Scientific Instrument,2013, 34(8): 1847-1854. (焉晓贞, 罗清华. 基于卡尔曼滤波的动态传感数据流估计方法[J]. 仪器仪表学报,2013, 34(8): 1847-1854.)
[20] XU Weiping, ZHU Qing, ZHANG Yeting, et al. Real-time GIS and its Application in Indoor Fire Disaster[C]//International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences. Istanbul: [s.n.],2013, XL-2/W2: 121-127.
