As the battle against COVID-19 continues, Stony Brook has been at the forefront of the pandemic with a convergence approach that includes both medicine and artificial intelligence. In Stony Brook's Institute for AI-Driven Discovery and Innovation, a multidisciplinary team is collaborating to develop a new way to look at image progression for COVID-19 patients, that helps the battle against the pandemic. In turn, Stony Brook faculty in the College of Engineering and Applied Sciences have made significant inroads that examine ways to fight the pandemic.

The multi-disciplinary team, including Aishik Konwer of the Department of Computer Science, and Joseph Bae, Chao Chen, Joel Saltz, and Prateek Prasanna of the Department of Biomedical Informatics, and collaborating institutions: Newark Beth Israel Medical Center and University Hospitals Cleveland Medical Center have developed a predictive deep-learning network for image progression from serial chest radiographs of COVID-19 patients. This work is reported in their paper: “Predicting COVID-19 Lung Infiltrate Progression on Chest Radiographs Using Spatio-temporal LSTM based Encoder-Decoder Network.”

SARS CoV-2 has infected 150 million people and claimed the lives of over three million people worldwide. Because of this, the dominating clinical motivation of this research was to improve health outcomes for those inflicted by the virus. In order to execute this, these Stony Brook faculty have focused on serial chest radiographs of COVID-19 inpatients.

“One of the motivations of this model was to understand the imaging progression of COVID-19 through the use of machine learning,” says Prasanna.

In chest images, deep learning-driven studies being conducted on COVID-19 typically employ one-shot imaging from a single time point. A shortcoming of this approach is that it does not accurately display disease progression.

Through the use of a variety of chest radiographs over an array of timepoints, the team at Stony Brook, has been able to more precisely model the development of the disease.

Artificial intelligence powers this novel approach. Sequential chest radiographs serve as the input to this network. Then, convolutional neural networks contextualize observable characteristics of the concerned lung zone. A two-stage spatio-temporal Long Short Term Memory based architecture is further incorporated into this model, which produces a deep learning framework to track and predict severity progression.

By tracking the evolution of the disease, “we can find a use in understanding the efficacy of procedures, such as proning,” says Prasanna. “This technology can be analyzed to provide insights into when proning (laying a patient on their abdomen to improve breathing) or other treatments should be initiated and for what duration.”

Together, these Stony Brook faculty are working on “further developing this framework towards understanding long-term sequelae of COVID-19,” says Prasanna. As this research continues, Stony Brook University carries on to make our mark in the fight against the global COVID-19 pandemic. “This work is only the beginning.”

This outstanding work is about to be presented at the conference for Medical Imaging and Deep Learning (MIDL 2021).

Research was enabled by the Renaissance School of Medicine at Stony Brook University’s “COVID-19 Data Commons and Analytic Environment," a data quality initiative instituted by the Office of the Dean, and supported by the Department of Biomedical Informatics. 

The Departments of Biomedical Informatics and Computer Science are part of Stony Brook University’s College of Engineering and Applied Sciences.