Multi-modal learning in Chest X-rays
Principal Investigator
Name
Benjamin Hou
Degrees
PhD
Institution
Imperial College London
Position Title
Research Associate
Email
About this CDAS Project
Study
PLCO
(Learn more about this study)
Project ID
PLCOI-774
Initial CDAS Request Approval
Apr 19, 2021
Title
Multi-modal learning in Chest X-rays
Summary
Chest radiographs are one of the most common diagnostic modalities in clinical routine. It can be done cheaply, requires minimal equipment, and the image can be diagnosed by every radiologists. However, the number of chest radiographs obtained on a daily basis can easily overwhelm the available clinical capacities. Clinical report writing is a tedious, highly variable and time consuming task. In countries where the population is large and radiologists are in high demand, a single radiologist may be required to read hundreds of radiology images and/or reports per day.
We seek to develop a model that can perform automatic report generation for chest radiographs with high clinical accuracy. Our intention is not to replace radiologists, but instead to assist them in their day-to-day clinical duties by accelerating the process of report generation. Current large scale datasets uses labels that are mined using Natural Language Processing (NLP) methods. Furthermore, the labels between radiological reports and images have large amounts of disagreements. This hinders the model's learning process and prevents it from performing to its full potential.
We seek to develop a model that can perform automatic report generation for chest radiographs with high clinical accuracy. Our intention is not to replace radiologists, but instead to assist them in their day-to-day clinical duties by accelerating the process of report generation. Current large scale datasets uses labels that are mined using Natural Language Processing (NLP) methods. Furthermore, the labels between radiological reports and images have large amounts of disagreements. This hinders the model's learning process and prevents it from performing to its full potential.
Aims
- Use machine learning to investigate if exists a shared latent space between chest X-ray images and chest X-ray reports
- Investigate if transformer networks are able to generate accurate radiograph reports
- Combined with radiograph reports and images into a multi modal model to improve pathological disease prediction
- Thus overall improving / accelerating patient care outcomes
Collaborators
Bernhard Kainz, Imperial College London
Jeremy Tan, Imperial College London