Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet_Journal of Biomedical Engineering

Authors：

 FENG Guohong , ZHENG Xiao , ZHANG Bin , WANG Hongen

College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin 150040, P. R. China;

Corresponding author：

FENG Guohong, Email: fgh_1980@126.com

Keywords：

Electroencephalogram; Emotion recognition; Independent component analysis; Recurrence plot; EfficientNet; Attention mechanism

DOI：

10.7507/1001-5515.202406029

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

To accurately capture and effectively integrate the spatiotemporal features of electroencephalogram (EEG) signals for the purpose of improving the accuracy of EEG-based emotion recognition, this paper proposes a new method combining independent component analysis-recurrence plot with an improved EfficientNet version 2 (EfficientNetV2). First, independent component analysis is used to extract independent components containing spatial information from key channels of the EEG signals. These components are then converted into two-dimensional images using recurrence plot to better extract emotional features from the temporal information. Finally, the two-dimensional images are input into an improved EfficientNetV2, which incorporates a global attention mechanism and a triplet attention mechanism, and the emotion classification is output by the fully connected layer. To validate the effectiveness of the proposed method, this study conducts comparative experiments, channel selection experiments and ablation experiments based on the Shanghai Jiao Tong University Emotion Electroencephalogram Dataset (SEED). The results demonstrate that the average recognition accuracy of our method is 96.77%, which is significantly superior to existing methods, offering a novel perspective for research on EEG-based emotion recognition.

Citation： FENG Guohong, ZHENG Xiao, ZHANG Bin, WANG Hongen. Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet. Journal of Biomedical Engineering, 2024, 41(6): 1103-1109. doi: 10.7507/1001-5515.202406029 Copy

1.	Hamada M, Zaidan B B, Zaidan A A. A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. Journal of Medical Systems, 2018, 42(9): 162.
2.	Liu S, Wang Z, An Y, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network. Knowledge-Based Systems, 2023, 265: 110372.
3.	Li M, Xu H, Liu X, et al. Emotion recognition from multichannel EEG signals using K-nearest neighbor classification. Technology and Health Care, 2018, 26(S1): 509-519.
4.	Yan C, Chang X, Li Z, et al. ZeroNAS: Differentiable generative adversarial networks search for zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9733-9740.
5.	Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, 2014, 5(3): 327-339.
6.	Duan R N, Zhu J Y, Lu B L. Differential entropy feature for EEG-based emotion classification//2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego: IEEE, 2013: 81-84.
7.	Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation, 2018, 10: 368-380.
8.	Cheng C, Yu Z, Zhang Y, et al. Hybrid network using dynamic graph convolution and temporal self-attention for EEG-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems, 2023: 2162-2388.
9.	Rahman M A, Anjum A, Milu M M H, et al. Emotion recognition from EEG-based relative power spectral topography using convolutional neural network. Array, 2021, 11: 100072.
10.	Wang Z, Wang Y, Zhang J, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
11.	王伟, 周建华, 刘紫恒, 等. 基于时空Inception残差注意力网络的脑电情绪识别. 重庆邮电大学学报(自然科学版), 2024, 36(1): 68-75.
12.	崔发昌. 时-频-空多维融合的脑电情绪分类方法. 南京: 南京邮电大学, 2023.
13.	宋昊, 徐颂, 刘国明, 等. 基于独立成分分析的非侵入式脑-机接口眼电伪迹自动去除算法. 生物医学工程学杂志, 2022, 39(6): 1074-1081.
14.	Jiang W, Zhang D, Ling L, et al. Time series classification based on image transformation using feature fusion strategy. Neural Processing Letters, 2022, 54(5): 3727-3748.
15.	冯国红, 王宏恩, 刁鹏飞, 等. 小样本下基于递归图和迁移学习的轴承故障诊断. 机床与液压, 2024, 52(16): 240-248.
16.	Al Moteri M, Mahesh T R, Thakur A, et al. Enhancing accessibility for improved diagnosis with modified EfficientNetV2-S and cyclic learning rate strategy in women with disabilities and breast cancer. Frontiers in Medicine, 2024, 11: 1373244.
17.	Mouhcine K, Zrira N, Elafi I, et al. STEFF: spatio-temporal EfficientNet for dynamic texture classification in outdoor scenes. Heliyon, 2024, 10(3): e25360.
18.	Zhao Y, Ju Z, Sun T, et al. TGC-YOLOv5: an enhanced YOLOv5 drone detection model based on transformer, GAM & CA attention mechanism. Drones, 2023, 7(7): 446.
19.	仲峥迪, 屠良平, 冯雪琦, 等. EfficientNetV2-S-Triplet 7: 一种改进的星系形态学分类算法. 天文学报, 2024, 65(2): 55-68.
20.	Cui L, Dong Z, Xu H, et al. Triplet attention-enhanced residual tree-inspired decision network: a hierarchical fault diagnosis model for unbalanced bearing datasets. Advanced Engineering Informatics, 2024, 59: 102322.
21.	聂聃, 王晓韡, 段若男, 等. 基于脑电的情绪识别研究综述. 中国生物医学工程学报, 2012, 31(4): 595-606.
22.	Zheng W L, Lu B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
23.	焦凯强, 王湖斐, 郭茂田. 脑电信号中的频谱不对称指数特征与情绪识别. 科学技术与工程, 2018, 18(17): 145-149.
24.	王雪蒙, 郭滨, 马欣. 基于优化变分模态分解的脑电情绪识别. 计算机应用与软件, 2024, 41(2): 80-85,177.
25.	Zhang G, Davoodnia V, Sepas-Moghaddam A, et al. Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sensors Journal, 2019, 20(6): 3113-3122.
26.	张冰雪, 李文楷. 少量通道脑电信号的实时情绪分类模型. 小型微型计算机系统, 2024, 45(2): 271-277.
27.	戴紫玉,马玉良, 高云园, 等. 基于多尺度卷积核CNN的脑电情绪识别. 传感技术学报, 2021, 34(4): 496-503.
28.	Yang C J, Li W C, Wan M T, et al. Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network//2021 IEEE International Symposium on Circuits and Systems (ISCAS). Daegu: IEEE, 2021: 1-5.
29.	谷学静, 刘佳, 郭宇承, 等. 采用多尺度多路混合注意力机制的脑电情绪识别方法. 计算机工程与应用, 2024, 60(19): 130-138.
30.	刘柯, 张孝, 李沛洋, 等. 基于脑功能网络和共空间模式分析的脑电情绪识别. 计算机应用研究, 2021, 38(5): 1344-1349.
31.	高越, 傅湘玲, 欧阳天雄, 等. 基于时空自适应图卷积神经网络的脑电信号情绪识别. 计算机科学, 2022, 49(4): 30-36.
32.	李奇, 常立娜, 武岩, 等. 基于深层图卷积的EEG情绪识别方法研究. 电子测量技术, 2024, 47(4): 18-22.
33.	谢松云, 雷凌俊, 孙江, 等. 基于IWOA-ELM算法的脑电情绪识别方法研究. 生物医学工程学杂志, 2024, 41(1): 1-8.

1. Hamada M, Zaidan B B, Zaidan A A. A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. Journal of Medical Systems, 2018, 42(9): 162.
2. Liu S, Wang Z, An Y, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network. Knowledge-Based Systems, 2023, 265: 110372.
3. Li M, Xu H, Liu X, et al. Emotion recognition from multichannel EEG signals using K-nearest neighbor classification. Technology and Health Care, 2018, 26(S1): 509-519.
4. Yan C, Chang X, Li Z, et al. ZeroNAS: Differentiable generative adversarial networks search for zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9733-9740.
5. Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, 2014, 5(3): 327-339.
6. Duan R N, Zhu J Y, Lu B L. Differential entropy feature for EEG-based emotion classification//2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego: IEEE, 2013: 81-84.
7. Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation, 2018, 10: 368-380.
8. Cheng C, Yu Z, Zhang Y, et al. Hybrid network using dynamic graph convolution and temporal self-attention for EEG-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems, 2023: 2162-2388.
9. Rahman M A, Anjum A, Milu M M H, et al. Emotion recognition from EEG-based relative power spectral topography using convolutional neural network. Array, 2021, 11: 100072.
10. Wang Z, Wang Y, Zhang J, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
11. 王伟, 周建华, 刘紫恒, 等. 基于时空Inception残差注意力网络的脑电情绪识别. 重庆邮电大学学报(自然科学版), 2024, 36(1): 68-75.
12. 崔发昌. 时-频-空多维融合的脑电情绪分类方法. 南京: 南京邮电大学, 2023.
13. 宋昊, 徐颂, 刘国明, 等. 基于独立成分分析的非侵入式脑-机接口眼电伪迹自动去除算法. 生物医学工程学杂志, 2022, 39(6): 1074-1081.
14. Jiang W, Zhang D, Ling L, et al. Time series classification based on image transformation using feature fusion strategy. Neural Processing Letters, 2022, 54(5): 3727-3748.
15. 冯国红, 王宏恩, 刁鹏飞, 等. 小样本下基于递归图和迁移学习的轴承故障诊断. 机床与液压, 2024, 52(16): 240-248.
16. Al Moteri M, Mahesh T R, Thakur A, et al. Enhancing accessibility for improved diagnosis with modified EfficientNetV2-S and cyclic learning rate strategy in women with disabilities and breast cancer. Frontiers in Medicine, 2024, 11: 1373244.
17. Mouhcine K, Zrira N, Elafi I, et al. STEFF: spatio-temporal EfficientNet for dynamic texture classification in outdoor scenes. Heliyon, 2024, 10(3): e25360.
18. Zhao Y, Ju Z, Sun T, et al. TGC-YOLOv5: an enhanced YOLOv5 drone detection model based on transformer, GAM & CA attention mechanism. Drones, 2023, 7(7): 446.
19. 仲峥迪, 屠良平, 冯雪琦, 等. EfficientNetV2-S-Triplet 7: 一种改进的星系形态学分类算法. 天文学报, 2024, 65(2): 55-68.
20. Cui L, Dong Z, Xu H, et al. Triplet attention-enhanced residual tree-inspired decision network: a hierarchical fault diagnosis model for unbalanced bearing datasets. Advanced Engineering Informatics, 2024, 59: 102322.
21. 聂聃, 王晓韡, 段若男, 等. 基于脑电的情绪识别研究综述. 中国生物医学工程学报, 2012, 31(4): 595-606.
22. Zheng W L, Lu B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
23. 焦凯强, 王湖斐, 郭茂田. 脑电信号中的频谱不对称指数特征与情绪识别. 科学技术与工程, 2018, 18(17): 145-149.
24. 王雪蒙, 郭滨, 马欣. 基于优化变分模态分解的脑电情绪识别. 计算机应用与软件, 2024, 41(2): 80-85,177.
25. Zhang G, Davoodnia V, Sepas-Moghaddam A, et al. Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sensors Journal, 2019, 20(6): 3113-3122.
26. 张冰雪, 李文楷. 少量通道脑电信号的实时情绪分类模型. 小型微型计算机系统, 2024, 45(2): 271-277.
27. 戴紫玉,马玉良, 高云园, 等. 基于多尺度卷积核CNN的脑电情绪识别. 传感技术学报, 2021, 34(4): 496-503.
28. Yang C J, Li W C, Wan M T, et al. Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network//2021 IEEE International Symposium on Circuits and Systems (ISCAS). Daegu: IEEE, 2021: 1-5.
29. 谷学静, 刘佳, 郭宇承, 等. 采用多尺度多路混合注意力机制的脑电情绪识别方法. 计算机工程与应用, 2024, 60(19): 130-138.
30. 刘柯, 张孝, 李沛洋, 等. 基于脑功能网络和共空间模式分析的脑电情绪识别. 计算机应用研究, 2021, 38(5): 1344-1349.
31. 高越, 傅湘玲, 欧阳天雄, 等. 基于时空自适应图卷积神经网络的脑电信号情绪识别. 计算机科学, 2022, 49(4): 30-36.
32. 李奇, 常立娜, 武岩, 等. 基于深层图卷积的EEG情绪识别方法研究. 电子测量技术, 2024, 47(4): 18-22.
33. 谢松云, 雷凌俊, 孙江, 等. 基于IWOA-ELM算法的脑电情绪识别方法研究. 生物医学工程学杂志, 2024, 41(1): 1-8.

Journal of Biomedical Engineering

Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content