面向行人导航意图探测的脑电分类研究

doi:10.11947/j.AGCS.2024.20230444

摘要/Abstract

摘要：

行人导航意图的自动识别是行人导航研究的一个难点问题，对建立智慧导航服务与新型的人机交互方式至关重要。目前，利用行为模式推估导航意图成为主流的解决方案，但是，这种方案依赖多种传感器且具有时滞性。本文提出了一种基于脑成像技术的行人导航意图探测方法，通过多导联的、高时间分辨率的脑电信号解译行人的转向意图。首先，在处于道路交叉口的场景下，依照标准的运动想象范式采集得到4类导航意图对应的脑电原始数据，包括直行、停止、左转和右转；然后，融合脑电在时频域、空间域与功能连接上的特征，构建表达脑电活动过程的脑电时空连接网络，便于捕获与导航意图高度相关的脑电特征；最后，采用图卷积神经网络编码脑电时空连接网络，完成由脑电到4类导航意图的映射，并利用9个被试者的脑电数据作为样本集对本文方法的有效性进行验证。试验结果表明，采用短时窗（1 s）探测4类导航意图的平均精度为0.443±0.062，最高精度可达0.571。采用长时窗（6 s）探测4类导航意图的平均精度为0.525±0.084，最高精度可达0.665。该方法的探测精度略优于其他脑电解译算法，且对前进和停止意图的识别能力优秀，最高可达0.740和0.700。

关键词: 行人导航, 导航意图识别, EEG, GCN

Abstract:

The automatic recognition of pedestrian intentions is a difficult issue in location-based services, which is crucial for establishing intelligent navigation services and new human-computer interaction method. Currently, using behavior patterns to estimate pedestrian intentions has become a mainstream solution, but this approach relies on multiple sensors and has time delays. This article proposes a pedestrian intention detection method based on brain imaging technology, which interprets pedestrian turning intentions through multi-channel, high-resolution EEG signals. Firstly, according to the standard motor imagery paradigm, EEG samples corresponding to four types of intentions within road intersection scenes were collected, including straight ahead, stop, left turn, and right turn. Then, by fusing the features of EEG in time-frequency domain, spatial domain, and functional connectivity domain, the spatiotemporal functional connectivity networks (STFCNs) of EEG are constructed to express the process of EEG activity, facilitating the capture of EEG features highly related to the intent. Finally, a graph convolutional neural network was used to encode the STFCNs, completing the mapping from EEG to four types of navigation intentions. The experimental results show that the average accuracy (F₁ score) of detecting four types of intentions using a short time window (1 s) is 0.443±0.062, and the highest accuracy can reach 0.571. The average accuracy with a long time window (6 s) is 0.525±0.084, and the highest accuracy is 0.665. The detection accuracy of this method is slightly better than other classification algorithms, and its detection ability for forward and stop intentions is excellent, up to 0.740 and 0.700.

Key words: pedestrian navigation, intention detection, EEG, GCN

中图分类号:

P228

方志祥, 王禄斌. 面向行人导航意图探测的脑电分类研究[J]. 测绘学报, 2024, 53(9): 1829-1841.

Zhixiang FANG, Lubin WANG. Detecting pedestrian intention using EEG signals in navigation[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(9): 1829-1841.

图/表 13

图1

图2

图3

图4

图5

图6

表1

图7

图8

图9

表2

图10

图11

参考文献 36

[1]	方志祥, 徐虹, 萧世伦, 等. 绝对空间定位到相对空间感知的行人导航研究趋势[J]. 武汉大学学报(信息科学版), 2018, 43(12):2173-2182.
	FANG Zhixiang, XU Hong, XIAO Shilun, et al. Pedestrian navigation research trend: from absolute space to relative space-based approach[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):2173-2182.
[2]	李德毅, 赵菲, 刘萌, 等. 自动驾驶量产的难点分析及展望[J]. 武汉大学学报(信息科学版), 2018, 43(12):1775-1779.
	LI Deyi, ZHAO Fei, LIU Meng, et al. Difficulty analysis and prospect of autonomous vehicle mass production[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1775-1779.
[3]	吕超, 崔格格, 孟相浩, 等. 基于图表示的智能车行人意图识别方法[J]. 北京理工大学学报, 2022, 42(7):688-695.
	LÜ Chao, CUI Gege, MENG Xianghao, et al. Graph representation method for pedestrian intention recognition of intelligent vehicle[J]. Transactions of Beijing Institute of Technology, 2022, 42(7):688-695.
[4]	方志祥, 罗浩, 李灵. 有限状态自动机辅助的行人导航状态匹配算法[J]. 测绘学报, 2017, 46(3):371-380. DOI: 10.11947/j.AGCS.2017.20160530.
	FANG Zhixiang, LUO Hao, LI Ling. A finite state machine aided pedestrian navigation state matching algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(3):371-380. DOI: 10.11947/j.AGCS.2017.20160530.
[5]	DOSHI A, TRIVEDI M M. On the roles of eye gaze and head dynamics in predicting driver's intent to change lanes[J]. IEEE Transactions on Intelligent Transportation Systems, 2009, 10(3):453-462.
[6]	SHI Bowen, DONG Weihua, ZHAN Zhicheng. AdaFI-FCN: an adaptive feature integration fully convolutional network for predicting driver's visual attention[J]. Geo-spatial Information Science, 2022, 27(4):1-17.
[7]	HEI Qiaosong, DONG Weihua, SHI Bowen. Detecting dynamic visual attention in augmented reality aided navigation environment based on a multi-feature integration fully convolutional network[J]. Cartography and Geographic Information Science, 2023, 50(1):63-78.
[8]	RAJWAL S, AGGARWAL S. Convolutional neural network-based EEG signal analysis: a systematic review[J]. Archives of Computational Methods in Engineering, 2023, 30(6):3585-3615.
[9]	WEI Ying, ZHOU Jun, WANG Yin, et al. A review of algorithm & hardware design for AI-based biomedical applications[J]. IEEE Transactions on Biomedical Circuits and Systems, 2020, 14(2):145-163.
[10]	KHALIFA Y, MANDIC D, SEJDIĆ E. A review of hidden Markov models and recurrent neural networks for event detection and localization in biomedical signals[J]. Information Fusion, 2021, 69:52-72.
[11]	ZABIHI S M, BEAUCHEMIN S S, BAUER M A. Real-time driving manoeuvre prediction using IO-HMM and driver cephalo-ocular behaviour[C]//Proceedings of 2017 IEEE Intelligent Vehicles Symposium. Los Angeles: IEEE, 2017: 875-880.
[12]	张杨松, 卓彦, 尧德中. 脑电磁成像进展及展望[J]. 中国科学：生命科学, 2020, 50(11):1268-1284.
	ZHANG Yangsong, ZHUO Yan, YAO Dezhong. Progresses and prospects of brain electromagnetic imaging[J]. Scientia Sinica Vitae, 2020, 50(11):1268-1284.
[13]	NICOLAS-ALONSO L F, GOMEZ-GIL J. Brain computer interfaces, a review[J]. Sensors (Basel, Switzerland), 2012, 12(2):1211-1279.
[14]	董卫华, 廖华, 詹智成, 等. 2008年以来地图学眼动与视觉认知研究新进展[J]. 地理学报, 2019, 74(3):599-614.
	DONG Weihua, LIAO Hua, ZHAN Zhicheng, et al. New research progress of eye tracking-based map cognition in cartography since 2008[J]. Acta Geographica Sinica, 2019, 74(3):599-614.
[15]	STOJIC F, CHAU T. Nonspecific visuospatial imagery as a novel mental task for online EEG-based BCI control[J]. International Journal of Neural Systems, 2020, 30(6):2050026.
[16]	SOUZA R H C E, NAVES E L M. Attention detection in virtual environments using EEG signals: a scoping review[J]. Frontiers in Physiology, 2021, 12:727840.
[17]	KOHLI V, TRIPATHI U, CHAMOLA V, et al. A review on virtual reality and augmented reality use-cases of brain computer interface based applications for smart cities[J]. Microprocessors and Microsystems, 2022, 88:104392.
[18]	WUNDERLICH A, GRAMANN K. Eye movement-related brain potentials during assisted navigation in real-world environments[J]. The European Journal of Neuroscience, 2021, 54(12):8336-8354.
[19]	LIN C T, KING J T, SINGH A K, et al. Voice navigation effects on real-world lane change driving analysis using an electroencephalogram[J]. IEEE Access, 2018, 6:26483-26492.
[20]	YU Yang, LIU Yadong, JIANG Jun, et al. An asynchronous control paradigm based on sequential motor imagery and its application in wheelchair navigation[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(12):2367-2375.
[21]	BOASHASH B, KHAN N A, BEN-JABEUR T. Time-frequency features for pattern recognition using high-resolution TFDs: a tutorial review[J]. Digital Signal Processing, 2015, 40:1-30.
[22]	王韬, 柯余峰, 王宁慈, 等. 空间滤波方法在脑-机接口中的应用及研究进展[J]. 中国生物医学工程学报, 2019, 38(5):599-608.
	WANG Tao, KE Yufeng, WANG Ningci, et al. Application and research development of spatial filtering method in brain-computer interfaces[J]. Chinese Journal of Biomedical Engineering, 2019, 38(5):599-608.
[23]	HE Bin, ASTOLFI L, VALDES-SOSA P A, et al. Electrophysiological brain connectivity: theory and implementation[J]. IEEE Transactions on Bio-Medical Engineering, 2019:2913928.
[24]	GRAMFORT A, LUESSI M, LARSON E, et al. MEG and EEG data analysis with MNE-Python[J]. Frontiers in Neuroscience, 2013, 7:267.
[25]	DHARMAPRANI D, NGUYEN H K, LEWIS T W, et al. A comparison of independent component analysis algorithms and measures to discriminate between EEG and artifact components[C]//Proceedings of 2016 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Orlando: IEEE, 2016: 825-828.
[26]	CAMPOS VIOLA F, THORNE J, EDMONDS B, et al. Semi-automatic identification of independent components representing EEG artifact[J]. Clinical Neurophysiology, 2009, 120(5):868-877.
[27]	DAMMERS J, SCHIEK M, BOERS F, et al. Integration of amplitude and phase statistics for complete artifact removal in independent components of neuromagnetic recordings[J]. IEEE Transactions on Biomedical Engineering, 2008, 55(10):2353-2362.
[28]	罗志增, 鲁先举, 周莹. 基于脑功能连接网络和样本熵的脑电信号特征提取[J]. 电子与信息学报, 2021, 43(2):412-418.
	LUO Zhizeng, LU Xianju, ZHOU Ying. EEG feature extraction based on brain function network and sample entropy[J]. Journal of Electronics & Information Technology, 2021, 43(2):412-418.
[29]	GROSSE-WENTRUP M, BUSS M. Multiclass common spatial patterns and information theoretic feature extraction[J]. IEEE Transactions on Biomedical Engineering, 2008, 55(8):1991-2000.
[30]	LI Xuan, WU Yunqiao, WEI Mengting, et al. A novel index of functional connectivity: phase lag based on Wilcoxon signed rank test[J]. Cognitive Neurodynamics, 2021, 15(4):621-636.
[31]	STAM C J, NOLTE G, DAFFERTSHOFER A. Phase lag index: assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources[J]. Human Brain Mapping, 2007, 28(11):1178-1193.
[32]	VINCK M, OOSTENVELD R, VAN WINGERDEN M, et al. An improved index of phase-synchronization for electrophysiological data in the presence of volume-conduction, noise and sample-size bias[J]. NeuroImage, 2011, 55(4):1548-1565.
[33]	HAMILTON WL, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: Curran Associates Inc, 2017: 1025-1035.
[34]	DIEHL F, BRUNNER T, LE TM, KNOLL A. Towards graph pooling by edge contraction[C]//Proceedings of 2019 ICML Workshop on Learning and Reasoning with Graph-structured Data. New York: ACM Press, 2019.
[35]	SU Zidong, HU Zehui, LI Yangding. Hierarchical graph representation learning with local capsule pooling[C]//Proceedings of 2021 ACM Multimedia Asia. Gold Coast: ACM Press, 2021: 1-7.
[36]	LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5):056013.

样本规模的组别	L/s	L_T/s	N_T	N_S
1	1	1	1	1920
2	3	1	5	640
3	3	2	2	640
4	3	3	1	640
5	6	1	11	320
6	6	2	5	320
7	6	4	2	320
8	6	6	1	320

模型	右转		左转		直行		停止
模型	精确率	召回率	精确率	召回率	精确率	召回率	精确率	召回率
EEGNet	50.6±10.2	49.3±12.8	49.4±12.3	49.7±10.4	53.7±9.3	54.1±12.3	54.5±11.8	55.1±11.7
GCN	50.4±9.1	48.6±11.6	47.4±14.1	48.4±9.9	55.3±10.4	55.9±13.6	55.4±14.4	56.1±13.5
STFCN_CSP	49.1±7.5	45.2±7.8	46.7±9.6	46.9±11.5	54.2±9.8	55.8±12.0	54.1±10.7	56.1±10.5
STFCN_Freq	35.9±11.6	35.1±9.7	36.8±9.0	35.6±9.9	37.4±10.2	38.4±10.3	37.6±9.4	36.3±8.6
MLP	46.6±10.7	45.2±10.4	46.7±8.6	44.8±10.4	49.2±9.7	48.3±10.7	50.2±10.8	49.6±10.7
本文模型	50.2±6.4	47.5±7.5	48.6±9.2	48.9±10.3	55.6±8.6	56.7±11.9	55.5±11.3	56.7±11.4