Cardiac magnetic resonance image segmentation based on lightweight network and knowledge distillation strategy_Journal of Biomedical Engineering

Authors：

LIU Zeqi , WANG Ning , ZHANG Chong ,  WEI Guohui

College of Intelligence and Information Engineering, Shandong University of Traditional Chinese Medicine, Jinan 250355, P. R. China;

Corresponding author：

WEI Guohui, Email: bmie530@163.com

Keywords：

Deep learning; Medical image segmentation; Knowledge distillation; Lightweight network

DOI：

10.7507/1001-5515.202312015

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

To address the issue of a large number of network parameters and substantial floating-point operations in deep learning networks applied to image segmentation for cardiac magnetic resonance imaging (MRI), this paper proposes a lightweight dilated parallel convolution U-Net (DPU-Net) to decrease the quantity of network parameters and the number of floating-point operations. Additionally, a multi-scale adaptation vector knowledge distillation (MAVKD) training strategy is employed to extract latent knowledge from the teacher network, thereby enhancing the segmentation accuracy of DPU-Net. The proposed network adopts a distinctive way of convolutional channel variation to reduce the number of parameters and combines with residual blocks and dilated convolutions to alleviate the gradient explosion problem and spatial information loss that might be caused by the reduction of parameters. The research findings indicate that this network has achieved considerable improvements in reducing the number of parameters and enhancing the efficiency of floating-point operations. When applying this network to the public dataset of the automatic cardiac diagnosis challenge (ACDC), the dice coefficient reaches 91.26%. The research results validate the effectiveness of the proposed lightweight network and knowledge distillation strategy, providing a reliable lightweighting idea for deep learning in the field of medical image segmentation.

Citation： LIU Zeqi, WANG Ning, ZHANG Chong, WEI Guohui. Cardiac magnetic resonance image segmentation based on lightweight network and knowledge distillation strategy. Journal of Biomedical Engineering, 2024, 41(6): 1204-1212. doi: 10.7507/1001-5515.202312015 Copy

1.	Wang R, Lei T, Cui R, et al. Medical image segmentation using deep learning: a survey. IET Image Processing, 2022, 16(5): 1243-1267.
2.	Celaya A, Actor J A, Muthusivarajan R, et al. PocketNet: a smaller neural network for medical image analysis. IEEE Transactions on Medical Imaging, 2023, 42(4): 1172-1184.
3.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany: Springer International Publishing, 2015: 234-241.
4.	Chen J, Lu Y, Yu Q, et al. Transunet: transformers make strong encoders for medical image segmentation. Arxiv Preprint, 2021, arxiv: 2102.04306.
5.	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need//31st Conference on Neural Information Processing Systems (NIPS2017), Long Beach: NIPS, 2017: 6000-6010.
6.	Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.
7.	Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 10012-10022.
8.	Zhou H Y, Guo J, Zhang Y, et al. nnFormer: volumetric medical image segmentation via a 3D transformer. IEEE Transactions on Image Processing, 2023, 32: 4036-4045.
9.	Zhang X, Zhou X, Lin M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 6848-6856.
10.	Howard A G, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. Arxiv Preprint, 2017, arxiv: 1704.04861.
11.	Valanarasu J M J, Oza P, Hacihaliloglu I, et al. Medical transformer: gated axial-attention for medical image segmentation//Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), Strasbourg: Springer International Publishing, 2021: 36-46.
12.	Wang H, Zhu Y, Green B, et al. Axial-deeplab: stand-alone axial-attention for panoptic segmentation//European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 108-126.
13.	沈瑜, 严源, 宋婧. 等. 基于并行轻量化卷积和多尺度融合的脑部磁共振图像配准. 生物医学工程学杂志, 2024, 41(2): 213-219.
14.	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. Arxiv Preprint, 2015, arxiv: 1503.02531.
15.	Xu K, Rui L, Li Y, et al. Feature normalized knowledge distillation for image classification//European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 664-680.
16.	Tung F, Mori G. Similarity-preserving knowledge distillation//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul: IEEE, 2019: 1365-1374.
17.	Tian Z, Chen P, Lai X, et al. Adaptive perspective distillation for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 1372-1387.
18.	Li K, Yu L, Wang S, et al. Towards cross-modality medical image segmentation with online mutual knowledge distillation//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1): 775-783.
19.	Dou Q, Liu Q, Heng P A, et al. Unpaired multi-modal segmentation via knowledge distillation. IEEE Transactions on Medical Imaging, 2020, 39(7): 2415-2425.
20.	Hu M, Maillard M, Zhang Y, et al. Knowledge distillation from multi-modal to mono-modal segmentation networks//Medical Image Computing and Computer Assisted Intervention (MICCAI 2020), Lima, Peru: Springer International Publishing, 2020: 772-781.
21.	Luo Y, Chen Z, Zhou S, et al. Self-distillation augmented masked autoencoders for histopathological image classification. Arxiv Preprint, 2022, arxiv: 2203.16983.
22.	You C, Zhou Y, Zhao R, et al. SimCVD: simple contrastive voxel-wise representation distillation for semi-supervised medical image segmentation. IEEE Transactions on Medical Imaging, 2022, 41(9): 2228-2237.
23.	Tragakis A, Kaul C, Murray-Smith R, et al. The fully convolutional transformer for medical image segmentation//2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI: IEEE, 2023: 3660-3669.
24.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
25.	Zhao Q, Zhong L, Xiao J, et al. Efficient multi-organ segmentation from 3D abdominal CT images with lightweight network and knowledge distillation. IEEE Transactions on Medical Imaging, 2023, 42(9): 2513-2523.
26.	Bernard O, Lalande A, Zotti C, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?. IEEE Transactions on Medical Imaging, 2018, 37(11): 2514-2525.
27.	Isensee F, Jaeger P F, Kohl S A A, et al. nnU-Net: a self-configuring Method for deep learning-based biomedical image segmentation. Nature Methods, 2021, 18(2): 203-211.
28.	Wang G, Luo X, Gu R, et al. PyMIC: a deep learning toolkit for annotation-efficient medical image segmentation. Computer Methods and Programs in Biomedicine, 2023, 231: 107398.
29.	Mehta S, Rastegari M, Caspi A, et al. Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation//15th European Conference on Computer Vision (ECCV), Munich: ECCV, 2018: 561-580.
30.	Sandler M, Howard A, Zhu M, et al. MobileNetV2: inverted residuals and linear bottlenecks// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 4510-4520.

1. Wang R, Lei T, Cui R, et al. Medical image segmentation using deep learning: a survey. IET Image Processing, 2022, 16(5): 1243-1267.
2. Celaya A, Actor J A, Muthusivarajan R, et al. PocketNet: a smaller neural network for medical image analysis. IEEE Transactions on Medical Imaging, 2023, 42(4): 1172-1184.
3. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany: Springer International Publishing, 2015: 234-241.
4. Chen J, Lu Y, Yu Q, et al. Transunet: transformers make strong encoders for medical image segmentation. Arxiv Preprint, 2021, arxiv: 2102.04306.
5. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need//31st Conference on Neural Information Processing Systems (NIPS2017), Long Beach: NIPS, 2017: 6000-6010.
6. Cao H, Wang Y, Chen J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.
7. Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 10012-10022.
8. Zhou H Y, Guo J, Zhang Y, et al. nnFormer: volumetric medical image segmentation via a 3D transformer. IEEE Transactions on Image Processing, 2023, 32: 4036-4045.
9. Zhang X, Zhou X, Lin M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 6848-6856.
10. Howard A G, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. Arxiv Preprint, 2017, arxiv: 1704.04861.
11. Valanarasu J M J, Oza P, Hacihaliloglu I, et al. Medical transformer: gated axial-attention for medical image segmentation//Medical Image Computing and Computer Assisted Intervention (MICCAI 2021), Strasbourg: Springer International Publishing, 2021: 36-46.
12. Wang H, Zhu Y, Green B, et al. Axial-deeplab: stand-alone axial-attention for panoptic segmentation//European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 108-126.
13. 沈瑜, 严源, 宋婧. 等. 基于并行轻量化卷积和多尺度融合的脑部磁共振图像配准. 生物医学工程学杂志, 2024, 41(2): 213-219.
14. Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. Arxiv Preprint, 2015, arxiv: 1503.02531.
15. Xu K, Rui L, Li Y, et al. Feature normalized knowledge distillation for image classification//European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 664-680.
16. Tung F, Mori G. Similarity-preserving knowledge distillation//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul: IEEE, 2019: 1365-1374.
17. Tian Z, Chen P, Lai X, et al. Adaptive perspective distillation for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 1372-1387.
18. Li K, Yu L, Wang S, et al. Towards cross-modality medical image segmentation with online mutual knowledge distillation//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1): 775-783.
19. Dou Q, Liu Q, Heng P A, et al. Unpaired multi-modal segmentation via knowledge distillation. IEEE Transactions on Medical Imaging, 2020, 39(7): 2415-2425.
20. Hu M, Maillard M, Zhang Y, et al. Knowledge distillation from multi-modal to mono-modal segmentation networks//Medical Image Computing and Computer Assisted Intervention (MICCAI 2020), Lima, Peru: Springer International Publishing, 2020: 772-781.
21. Luo Y, Chen Z, Zhou S, et al. Self-distillation augmented masked autoencoders for histopathological image classification. Arxiv Preprint, 2022, arxiv: 2203.16983.
22. You C, Zhou Y, Zhao R, et al. SimCVD: simple contrastive voxel-wise representation distillation for semi-supervised medical image segmentation. IEEE Transactions on Medical Imaging, 2022, 41(9): 2228-2237.
23. Tragakis A, Kaul C, Murray-Smith R, et al. The fully convolutional transformer for medical image segmentation//2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI: IEEE, 2023: 3660-3669.
24. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
25. Zhao Q, Zhong L, Xiao J, et al. Efficient multi-organ segmentation from 3D abdominal CT images with lightweight network and knowledge distillation. IEEE Transactions on Medical Imaging, 2023, 42(9): 2513-2523.
26. Bernard O, Lalande A, Zotti C, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?. IEEE Transactions on Medical Imaging, 2018, 37(11): 2514-2525.
27. Isensee F, Jaeger P F, Kohl S A A, et al. nnU-Net: a self-configuring Method for deep learning-based biomedical image segmentation. Nature Methods, 2021, 18(2): 203-211.
28. Wang G, Luo X, Gu R, et al. PyMIC: a deep learning toolkit for annotation-efficient medical image segmentation. Computer Methods and Programs in Biomedicine, 2023, 231: 107398.
29. Mehta S, Rastegari M, Caspi A, et al. Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation//15th European Conference on Computer Vision (ECCV), Munich: ECCV, 2018: 561-580.
30. Sandler M, Howard A, Zhu M, et al. MobileNetV2: inverted residuals and linear bottlenecks// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 4510-4520.

Journal of Biomedical Engineering

Cardiac magnetic resonance image segmentation based on lightweight network and knowledge distillation strategy

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content