Validation of Artificial Intelligence solution to personalize lung cancer screening intervals
Principal Investigator
Name
Rasika Rajapakshe
Degrees
Ph.D.
Institution
BC Cancer, part of the Provincial Health Services Authority
Position Title
Senior Medical Physicist
Email
About this CDAS Project
Study
NLST
(Learn more about this study)
Project ID
NLST-1458
Initial CDAS Request Approval
Aug 12, 2025
Title
Validation of Artificial Intelligence solution to personalize lung cancer screening intervals
Summary
A critical component of this project is the rigorous external validation of the proposed AI lung cancer risk prediction model. Recognizing that many AI algorithms fail during independent testing, this project has a dedicated plan to prove the model's real-world effectiveness and generalizability. The core of this process involves testing the final, trained AI model on a dataset that is entirely separate and different from the data used for its initial training and fine-tuning.
Specifically, the model—trained on Canadian data from British Columbia—will be validated using a high-quality subset of baseline low-dose CT (LDCT) scans from the U.S.-based National Lung Screening Trial (NLST). This process is designed to test the model's performance under a "domain shift," meaning it must accurately predict long-term cancer risk in a different patient population, collected with different equipment, and within a different healthcare system. Successfully validating the model on the NLST dataset will demonstrate its robustness and readiness for broader clinical application, confirming its ability to reliably identify low-risk individuals who can safely have their screening intervals extended.
Specifically, the model—trained on Canadian data from British Columbia—will be validated using a high-quality subset of baseline low-dose CT (LDCT) scans from the U.S.-based National Lung Screening Trial (NLST). This process is designed to test the model's performance under a "domain shift," meaning it must accurately predict long-term cancer risk in a different patient population, collected with different equipment, and within a different healthcare system. Successfully validating the model on the NLST dataset will demonstrate its robustness and readiness for broader clinical application, confirming its ability to reliably identify low-risk individuals who can safely have their screening intervals extended.
Aims
Test the AI Model: The final AI risk prediction model, will be applied to this completely new and unseen NLST dataset. The model will analyze the LDCT images and generate a risk score for each patient.
Evaluate Performance and Generalizability: The model's predictions will be compared against the actual long-term patient outcomes from the NLST data. Performance will be measured using standard statistical metrics, including the Area Under the Receiver-Operating Curve (AUC) and Uno's C-index, to determine how accurately the model distinguished between individuals who developed cancer and those who remained cancer-free.
Collaborators
Dr. Stephen Lam, BC Cancer
Dr. Rafael Meza, BC Cancer
Dr Calum MacAulay, BC Cancer
Dr. Mohamed Shehata, UBC-Okanagan