Multimodal Cardiothoracic Disease Prediction

University of Sheffield Collaborating Faculties: Faculty of Medicine, Dentistry and Health, Faculty of Social Science, and Faculty of Engineering

External Partner: Sheffield Teaching Hospitals NHS Foundation Trust

Overview: Our project aims to develop a sophisticated Artificial Intelligence (AI) system which can process multimodal, multi-vendor, multi-centre, and multi-pathophysiological cardiothoracic data, such as Chest Radiographs (CXR), Echocardiogram (ECG), Computed Tomography (CT), Magnetic Resonance Imaging (MRI) and Electronic Health Record (EHR), to segment and classify pathophysiological features and improve the diagnosis, prognosis, and therapeutic response prediction of Cardiothoracic Disease (CTD) such as Pulmonary Hypertension (PH), Chronic Obstructive Pulmonary Disease (COPD) and Abnormal Heart Rhythms (AHR) to a level at which advanced methods such as contrastive learning, foundation model, meta-learning, few-shot and zero-shot learning can successfully extract interpretable clinical parameters.

Motivation: Cardiothoracic disease refers to a variety of conditions that affect the heart and lungs, including coronary artery disease, heart failure, lung cancer, and diseases of the chest wall. These illnesses can impact the overall function and structure of the heart and lungs, and they often require specialised care, potentially including surgery. Despite substantial progress in medical technology, the early diagnosis of these diseases remains a challenge due to the complexity of disease development and the vague symptoms produced in the early stages. In our initial study, we will utilise the MIMIC datasets, which comprise various data modalities within a comprehensive open-access database. This dataset has been the foundation for high-quality research in areas ranging from intensive care and mortality prediction to disease classification in pathology. Additionally, we will use the in-house dataset, ASPIRE registry, which also offers multiple data modalities, to further test our model. In the short term, the project’s success could lead to improved diagnosis and monitoring of cardiothoracic diseases, potentially reducing the need for invasive procedures and facilitating personalised treatment plans. In the long term, our AI system could be adapted to more cardiovascular and respiratory diseases, revolutionising the approach to cardiothoracic medicine and benefiting countless patients worldwide.