Self-supervised multi-modal neural networks for predicting response to immunotherapy in lung cancer patients.
Principal Investigator
Name
Raquel Perez-Lopez
Degrees
M.D., Ph.D.
Institution
Vall d'Hebron Institute of Oncology
Position Title
Principal Investigator
Email
About this CDAS Project
Study
NLST
(Learn more about this study)
Project ID
NLST-950
Initial CDAS Request Approval
Sep 6, 2022
Title
Self-supervised multi-modal neural networks for predicting response to immunotherapy in lung cancer patients.
Summary
We will use the National Lung Screening Trial (NLST) data to pre-train a transformer model with Self-supervised learning. We expect that this will improve the classification performance for immunotherapy response in our in-house cohort.
Overview
Immune checkpoint inhibitors are one of the most promising anti-cancer treatment strategies. However, despite its excellent results, there is still a need for better predictive biomarkers for patient selection and stratification. Much of the discussion has been focused on molecular characterization by genomics or proteomics. However, artificial intelligence (AI) has been applied to routine medical images such as haematoxylin and eosin-stained (H&E) histopathology slides and CT scans to predict molecular subtypes and genetic features associated with treatment response, including CD8+ T-cells, PDL1 expression, microsatellite instability (MSI) or tumor mutational burden (TMB). In this study, we aim to build a multi-modal neural network combining CT images and H&E-stained histopathology images to identify responsive imaging phenotypes for immune-checkpoints inhibitors (ICIs) using the National Lung Screening Trial (NLST) cohort as pre-training datasets.
Overview
Immune checkpoint inhibitors are one of the most promising anti-cancer treatment strategies. However, despite its excellent results, there is still a need for better predictive biomarkers for patient selection and stratification. Much of the discussion has been focused on molecular characterization by genomics or proteomics. However, artificial intelligence (AI) has been applied to routine medical images such as haematoxylin and eosin-stained (H&E) histopathology slides and CT scans to predict molecular subtypes and genetic features associated with treatment response, including CD8+ T-cells, PDL1 expression, microsatellite instability (MSI) or tumor mutational burden (TMB). In this study, we aim to build a multi-modal neural network combining CT images and H&E-stained histopathology images to identify responsive imaging phenotypes for immune-checkpoints inhibitors (ICIs) using the National Lung Screening Trial (NLST) cohort as pre-training datasets.
Aims
Aim 1: To develop integrative deep learning models combining CT and H&E-stained histopathology images to predict response to immunotherapy in lung cancer patients on an in-house database
Aim 2: To self-supervised pre-train deep learning models on the NLST CT scan image collection (75,000 CT scans) to improve CT imaging feature extraction.
Aim 3: To pre-train multi-modal transformers combining CT and H&E-stained histopathology images on the NLST pathology image collection (450 patients)
Collaborators
Jakob Nikolas Kather, Prof. Dr. med. (Else Kroener Fresenius Center for Digital Health, Technical University Dresden)
Marta Ligero, M.Sc. (Vall d'Hebron Institute of Oncology)