Spatio-temporal deep learning with temporal attention for indeterminate lung nodule classification.

Authors

Farina B, Carbajo Benito R, Montalvo-García D, Bermejo-Peláez D, Maceiras LS, Ledesma-Carbayo MJ

Affiliations

• Biomedical Image Technologies, ETSI Telecomunicación, Madrid, 28040, Spain; Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto Salud Carlos III, Madrid, 28040, Spain. Electronic address: benito.farina@upm.es.
• Biomedical Image Technologies, ETSI Telecomunicación, Madrid, 28040, Spain; Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto Salud Carlos III, Madrid, 28040, Spain.
• Biomedical Image Technologies, ETSI Telecomunicación, Madrid, 28040, Spain; Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto Salud Carlos III, Madrid, 28040, Spain; SPOTLAB, Madrid, 28040, Spain.
• Department of Oncology, Clínica Universidad de Navarra, Pamplona, 31008, Navarra, Spain; Centro de Investigación Biomédica en Red de Enfermedades Respiratorias (CIBER-ES), Instituto Salud Carlos III, Madrid, 28040, Spain.
• Biomedical Image Technologies, ETSI Telecomunicación, Madrid, 28040, Spain; Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto Salud Carlos III, Madrid, 28040, Spain. Electronic address: mj.ledesma@upm.es.

Abstract

Lung cancer is the leading cause of cancer-related death worldwide. Deep learning-based computer-aided diagnosis (CAD) systems in screening programs enhance malignancy prediction, assist radiologists in decision-making, and reduce inter-reader variability. However, limited research has explored the analysis of repeated annual exams of indeterminate lung nodules to improve accuracy. We introduced a novel spatio-temporal deep learning framework, the global attention convolutional recurrent neural network (globAttCRNN), to predict indeterminate lung nodule malignancy using serial screening computed tomography (CT) images from the National Lung Screening Trial (NLST) dataset. The model comprises a lightweight 2D convolutional neural network for spatial feature extraction and a recurrent neural network with a global attention module to capture the temporal evolution of lung nodules. Additionally, we proposed new strategies to handle missing data in the temporal dimension to mitigate potential biases arising from missing time steps, including temporal augmentation and temporal dropout. Our model achieved an area under the receiver operating characteristic curve (AUC-ROC) of 0.954 in an independent test set of 175 lung nodules, each detected in multiple CT scans over patient follow-up, outperforming baseline single-time and multiple-time architectures. The temporal global attention module prioritizes informative time points, enabling the model to capture key spatial and temporal features while ignoring irrelevant or redundant information. Our evaluation emphasizes its potential as a valuable tool for the diagnosis and stratification of patients at risk of lung cancer.