Tushar Mehta

Parkland High School

Research Assistant

NLST (Learn more about this study)

NLST-1369

Dec 16, 2024

Multimodal Deep Learning Model for Lung Cancer Detection: Optimal Fusion of Radiological and Clinical Data for Precision Medicine

In this project several different Convolutional Neural Network (CNN) and transformer models as well as transfer learning approaches will be evaluated for extracting features from medical images (CT scans, X-rays etc..) combined with machine learning approaches for the clinical data to develop robust cancer detection and classification system. In the first step of this project, features associated with cancerous regions/cells in the radiology data are extracted. In the second step, the focus is to identify the clinical variable which serves as robust predictors for the onset, identification or type classification of cancerous growth. The unique next step is an optimal fusion of insights from these two data sets to develop a patient specific diagnosis and treatment plan. Validations studies will be conducted to test the accuracy of the multimodal deep learning models by predicting the target class of the records from the field trial.

Develop best ways to combine radiology (X-ray or CT) and individualized patient data (demographic, lifestyle, lab reports) to improve the accuracy and accessibility of early-stage cancer diagnostics in the context of personalized medicine.

Identify explainable features of the ML model to highlight the onset of cancer and correlate with the demographic/clinical data.