Real-time Observational Water Level Data Stream Online Filtering Method with Hydrological Changes Semantic Constraints

doi:10.11947/j.AGCS.2015.20140416

Acta Geodaetica et Cartographica Sinica ›› 2015, Vol. 44 ›› Issue (12): 1351-1358.doi: 10.11947/j.AGCS.2015.20140416

Previous Articles Next Articles

Real-time Observational Water Level Data Stream Online Filtering Method with Hydrological Changes Semantic Constraints

DING Yulin^1,2,3, ZHU Qing^1,4, HE Xiaobo^1,4, LIN Hui³, DU Zhiqiang⁵, ZHANG Yeting⁵, MIAO Shuangxi^1,4, YANG Xiaoxia⁶

1. State-Province Joint Engineering Laboratory of Spatial Information Technology of High-speed Rail Safety, Southwest Jiaotong University, Chengdu 611756, China;
2. Key Laboratory of Poyang Lake Wetland and Watershed Research, Ministry of Education, Jiangxi Normal University, Nanchang 330000, China;
3. Institute of Space and Earth Information Science, The Chinese University of Hong Kong, Hong Kong 999077, China;
4. Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China;
5. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China;
6. College of Earth Sciences, Chengdu University of Technology, Chengdu 610059

Received:2014-08-13 Revised:2015-06-08 Online:2015-12-20 Published:2016-01-04
Supported by:
The Collaborative Innovation Center for Major Ecological Security Issues of Jiangxi Province and Monitoring Implementation (No.JXS-EW-00);The National Natural Science Foundation of China (Nos.41101354;41201440);The National High-tech Research Program of China (863 Program)(No. 2013AA122301)

Abstract

Abstract: Irregular environmental changes and occasional instrument malfunctions have made noises and exceptions in observational data prominence. Therefore, before processing real-time water level data online, data cleaning is urgently needed to ensure data quality. Since traditional data filtering methods didn't take the data change pattern into consideration, these methods have encountered some severe problems, including the poor adaptability of filter model, the low estimation precision and prohibitively high calculation cost. To overcome these shortcomings, this paper presents a hydrological change semantics constrained online Kalman filtering method: creating dynamic semantic mapping between real-time data changing pattern and the rules of spatial-temporal hydrological process evolution; implementing the change semantic constrained Kalman filtering method to support the adaptive parameter optimization. Observational water level data streams of different precipitation scenarios are selected for testing. Experimental results prove that by means of this method, more accurate and reliable water level information can be available.

Key words: real-time observational water level data stream, change semantic constraint, Kalman filtering

CLC Number:

P237

DING Yulin, ZHU Qing, HE Xiaobo, LIN Hui, DU Zhiqiang, ZHANG Yeting, MIAO Shuangxi, YANG Xiaoxia. Real-time Observational Water Level Data Stream Online Filtering Method with Hydrological Changes Semantic Constraints[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(12): 1351-1358.

References

[1] DING Yulin, FAN Yida, DU Zhiqiang, et al. An Integrated Geospatial Information Service System for Disaster Management in China[J]. International Journal of Digital Earth,2015, 8(11): 918-945.
[2] DING Yulin, ZHU Qing, LIN Hui. An Integrated Virtual Geographic Environmental Simulation Framework: A Case Study of Flood Disaster Simulation[J]. Geo-spatial Information Science,2014, 17(4): 190-200.
[3] DING Yulin, DU Zhiqiang, ZHU Qing, et al. Adaptive Water Level Correction Algorithm for Flooding Analysis[J]. Acta Geodaetica et Cartographica Sinica,2013, 42(4): 546-553. (丁雨淋, 杜志强, 朱庆, 等. 洪水淹没分析中的自适应逐点水位修正算法[J]. 测绘学报,2013, 42(4): 546-553.)
[4] SUI D, GOODCHILD M. The Convergence of GIS and Social Media: Challenges for GIScience[J]. International Journal of Geographical Information Science,2011, 25(11): 1737-1748.
[5] O'REILLY C, GLUHAK A, IMRAN M A, et al. Anomaly Detection in Wireless Sensor Networks in a Non-stationary Environment[J]. IEEE Communications Surveys & Tutorials, 2014, 16(3): 1413-1432.
[6] SMITH D, TIMMS G, De SOUZA P, et al. A Bayesian Framework for the Automated Online Assessment of Sensor Data Quality[J]. Sensors,2012, 12(7): 9476-9501.
[7] RASSAM M A, MAAROF M A, ZAINAL A. Adaptive and Online Data Anomaly Detection for Wireless Sensor Systems[J]. Knowledge-based Systems,2014, 60: 44-57.
[8] CHANDOLA V, BANERJEE A, KUMAR V. Anomaly Detection: A Survey[J]. ACM Computing Surveys,2009, 41(3): 15.
[9] HILL D J. Automated Bayesian Quality Control of Streaming Rain Gauge Data[J]. Environmental Modelling & Software,2013, 40: 289-301.
[10] PATCHA A, PARK J M. An Overview of Anomaly Detection Techniques: Existing Solutions and Latest Technological Trends[J]. Computer Networks,2007, 51(12): 3448-3470.
[11] LIU Hancong, SHAH S, JIANG Wei. On-line Outlier Detection and Data Cleaning[J]. Computers &Chemical Engineering,2004, 28(9): 1635-1647.
[12] INGLEBY B, HUDDLESTON M. Quality Control of Ocean Temperature and Salinity Profiles-historical and Real-time Data[J]. Journal of Marine Systems,2007, 65(1-4): 158-175.
[13] GELMAN A, CARLIN J B, STERN H S, et al. Bayesian Data Analysis[M]. 3rd ed. Boca Raton: Chapman and Hall/CRC press, 2013.
[14] TROYANSKAYA O G, DOLINSKI K, OWEN A B, et al. A Bayesian Framework for Combining Heterogeneous Data Sources for Gene Function Prediction (In Saccharomyces Cerevisiae)[C]//Proceedings of the National Academy of Sciences of the United States of America,2003, 100(14): 8348-8353.
[15] KANO M,NAKAGAWA Y. Data-based Process Monitoring, Process Control, and Quality Improvement: Recent Developments and Applications in Steel Industry[J]. Computers & Chemical Engineering,2008, 32(1-2): 12-24.
[16] LIU Fanming,QIAN Dong, GUO Jing. Terrain Estimation Algorithm Based on Kalman Filter and Its Simulation Research[J]. Acta Geodaetica et Cartographica Sinica,2011, 40(1): 45-51. (刘繁明, 钱东, 郭静. 基于卡尔曼滤波的地形反演方法及其仿真研究[J]. 测绘学报,2011, 40(1): 45-51.)
[17] LIU Guolin, HAO Huadong, YAN Man, et al. Phase Unwrapping Algorithm by Using Kalman Filter Based on Topographic Factors[J]. Acta Geodaetica et Cartographica Sinica,2011, 40(3): 283-288. (刘国林, 郝华东, 闫满, 等. 顾及地形因素的卡尔曼滤波相位解缠算法[J]. 测绘学报,2011, 40(3): 283-288.)
[18] LI Lihua, PENG Junhuan. Multiple Kalman Filters Model with Shaping Filter GPS Real-time Deformation Analysis[J]. Transactions of Nonferrous Metals Society of China,2014, 24(11): 3674-3681.
[19] YAN Xiaozhen, LUO Qinghua. Dynamic Sensor Data Stream Estimation Method Based on Kalman Filtering[J]. Chinese Journal of Scientific Instrument,2013, 34(8): 1847-1854. (焉晓贞, 罗清华. 基于卡尔曼滤波的动态传感数据流估计方法[J]. 仪器仪表学报,2013, 34(8): 1847-1854.)
[20] XU Weiping, ZHU Qing, ZHANG Yeting, et al. Real-time GIS and its Application in Indoor Fire Disaster[C]//International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences. Istanbul: [s.n.],2013, XL-2/W2: 121-127.

Real-time Observational Water Level Data Stream Online Filtering Method with Hydrological Changes Semantic Constraints

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 2

Recommended Articles

Metrics

Comments

[1]	LI Bofeng, QIN Yuanyang, CHEN Guang'e. BDS-3 cycle slip and data gap repair based on the geometry-free ionosphere-filter model [J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4): 501-510.
[2]	JIA Song, XU Tianhe, YANG Honglei. Two Improved Algorithms for LS+AR Prediction Model of the Polar Motion [J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(S0): 71-77.