Final Report

Speeding Up Examination of Heart Diseases with 3D Echocardiography and Machine Learning

View Full Text in PDF

Abstract

Modern hospitals use 2D echocardiography to obtain cross-section views of the heart. However, this process is time-consuming regarding the large number of cross-section views needed to be obtained and the insufficient training of transducer operators. The lengthy acquisition time also increases the wait time before a patient can receive an echocardiography examination. Therefore, the project aims to develop a machine learning algorithm to reconstruct the cross-section views from 3D echocardiography automatically. It consists of mainly two tasks. The first landmark localization task utilizes 3D echocardiography to shorten the acquisition time of the heart’s structural information and extends the fully convolutional SpatialConfiguration-Net (SCN) to detect local and global features of cardiac landmarks in the 3D echocardiography. SCN excels at this job due to its two task-oriented components that can be trained in an end-to-end manner. Besides, the Adaptive Wing loss is incorporated into SCN for better convergence to the heatmap-based supervision. The second cross-section recovery task adopts the least squared distance fit to recover the cross-section plane parameters based on the predicted landmark locations and reconstructs the 2D cross-section views. The project achieved desirable results in both the landmark localization and cross-section recovery tasks by demonstrating small errors on multiple metrics and outputting satisfactory visualizations on the cardiac dataset with limited training data. As a result, the acquisition and analyzing time will be significantly shortened. The project will be the world’s first AI software for automatically reconstructing cross-section views from 3D echocardiography. Furthermore, it is the prerequisite of many downstream 2D-based heart disease classifiers that can assist doctors in heart disease diagnoses and make mass heart disease screening possible.

Keywords: 3D echocardiography, convolutional neural network, landmark localization

Methodology

SpatialConfiguration-Net Architecture

The local appearance component generates heatmaps from the input echo data, and the spatial configuration component generates heatmaps from the local appearance heatmaps. The final heatmaps are the product of the local appearance and spatial configuration heatmaps. ${\color{YellowGreen}\square}$: input echo data, ${\color{CornflowerBlue}\square}$: local appearance heatmaps, ${\color{Red}\square}$: spatial configuration heatmaps, ${\color{Purple}\square}$: output heatmaps, ${\color{Gray}\square}$: intermediate feature maps; ${\color{YellowGreen}\rightarrow}$: convolution, ${\color{CornflowerBlue}\rightarrow}$: down-sampling, ${\color{Red}\rightarrow}$: up-sampling; ${\color{Orange}\oplus}$: element-wise addition, ${\color{Purple}\otimes}$: element-wise multiplication.

Evaluation

Baseline Comparison of Planar Position

$E_{realshift}$: the Euclidean distance from the centroid of the ground truth landmarks to the centroid of the predicted landmarks on a cross-section plane.

Baseline Comparison of Planar Orientation

$E_{angle}$: the angle difference of the ground truth cross-section plane and the predicted cross-section plane.

Visualization

Cross-section Visualization Examples

$\rm (P)$: predicted cross-sections, $\rm (T)$: ground truth cross-sections. The column header is the identifier of the echo data.

— Apr 19, 2023

Final Report by Lu Meng is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Permissions beyond the scope of this license may be available at About.