• University of Groningen, University Medical Center Groningen Groningen, Department of Epidemiology, Groningen, The Netherlands; Department of Pulmonary Medicine, Medisch Spectrum Twente, Enschede, The Netherlands. Electronic address: m.a.heuvelmans@umcg.nl.
• University of Groningen, University Medical Center Groningen Groningen, Department of Radiation Oncology, Groningen, The Netherlands. Electronic address: p.m.a.van.ooijen@umcg.nl.
• Department of Radiology, Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom. Electronic address: Sarim.Ather@ouh.nhs.uk.
• Diagnostic and Interventional Radiology, University Hospital Heidelberg, Heidelberg, Germany; Translational Lung Research Center, Member of the German Lung Research Center, Heidelberg, Germany; Department of Diagnostic and Interventional Radiology With Nuclear Medicine, Thoraxklinik, Heidelberg University Hospital, Heidelberg, Germany. Electronic address: Carlos.DaSilva@med.uni-heidelberg.de.
• University of Groningen, University Medical Center Groningen Groningen, Department of Radiology, Groningen, The Netherlands. Electronic address: d.han@umcg.nl.
• Diagnostic and Interventional Radiology, University Hospital Heidelberg, Heidelberg, Germany; Translational Lung Research Center, Member of the German Lung Research Center, Heidelberg, Germany; Department of Diagnostic and Interventional Radiology With Nuclear Medicine, Thoraxklinik, Heidelberg University Hospital, Heidelberg, Germany. Electronic address: heussel_elsevier2019@contbay.com.
• Oxford University Hospitals NHS Foundation Trust, United Kingdom. Electronic address: William.Hickes@ouh.nhs.uk.
• Diagnostic and Interventional Radiology, University Hospital Heidelberg, Heidelberg, Germany; Translational Lung Research Center, Member of the German Lung Research Center, Heidelberg, Germany; Department of Diagnostic and Interventional Radiology With Nuclear Medicine, Thoraxklinik, Heidelberg University Hospital, Heidelberg, Germany. Electronic address: Hans-Ulrich.Kauczor@med.uni-heidelberg.de.
• Optellum Ltd., Oxford, United Kingdom; Department of Respiratory Sciences, College of Life Sciences, University of Leicester, Leicester, United Kingdom. Electronic address: pnovotnyq@gmail.com.
• Department of Radiology, Oxford University Hospitals NHS Foundation Trust, Oxford, UK. Electronic address: Heiko.Peschl@ouh.nhs.uk.

INTRODUCTION: Deep Learning has been proposed as promising tool to classify malignant nodules. Our aim was to retrospectively validate our Lung Cancer Prediction Convolutional Neural Network (LCP-CNN), which was trained on US screening data, on an independent dataset of indeterminate nodules in an European multicentre trial, to rule out benign nodules maintaining a high lung cancer sensitivity.

METHODS: The LCP-CNN has been trained to generate a malignancy score for each nodule using CT data from the U.S. National Lung Screening Trial (NLST), and validated on CT scans containing 2106 nodules (205 lung cancers) detected in patients from from the Early Lung Cancer Diagnosis Using Artificial Intelligence and Big Data (LUCINDA) study, recruited from three tertiary referral centers in the UK, Germany and Netherlands. We pre-defined a benign nodule rule-out test, to identify benign nodules whilst maintaining a high sensitivity, by calculating thresholds on the malignancy score that achieve at least 99 % sensitivity on the NLST data. Overall performance per validation site was evaluated using Area-Under-the-ROC-Curve analysis (AUC).

RESULTS: The overall AUC across the European centers was 94.5 % (95 %CI 92.6-96.1). With a high sensitivity of 99.0 %, malignancy could be ruled out in 22.1 % of the nodules, enabling 18.5 % of the patients to avoid follow-up scans. The two false-negative results both represented small typical carcinoids.

CONCLUSION: The LCP-CNN, trained on participants with lung nodules from the US NLST dataset, showed excellent performance on identification of benign lung nodules in a multi-center external dataset, ruling out malignancy with high accuracy in about one fifth of the patients with 5-15 mm nodules.