Middle-aged woman with short auburn hair sits in front of a computer. She wears of professional attire. A building and flours can be seen through the window in the background.

UMBC’s Tinoosh Mohsenin develops COVID-Matter framework to determine severity of respiratory disease

When COVID-19 upended daily life a year and a half ago, scientists and engineers worldwide responded with new research on detecting, tracking, and managing cases. UMBC’s Tinoosh Mohsenin, associate professor of computer science and electrical engineering, has partnered with Mohammad Sajadi, associate professor at the Institute of Human Virology at University of Maryland, Baltimore (UMB) to develop COVID-Matter. It’s a technology to identify respiratory disease, associated symptoms, and their severity. 

COVID-Matter collects a broad range of complex physiological data and then applies machine learning techniques to understand the data and assess the severity of a patient’s respiratory disease. The data collected include the sound and frequency of a patient’s speech, cough, and breathing sounds.

From these sounds, the tool can extract important information, such as breathing rate, used to identify shortness of breath. The framework can also include facial recognition and patients’ reported levels of fatigue and confusion, to further improve its accuracy.

Two men and one woman stand in front of plants and a walking ramp, next to a sign reading "Information Technology/Engineering Building."
Tinoosh Mohsenin, center, with her two Ph.D. students who are working on COVID-Matter. Photo courtesy of Mohsenin.

Real-time, accessible solutions

The research team recently published a peer-reviewed conference paper describing their reconfigurable software-hardware machine learning framework for automatic detection of respiratory symptoms. They’ve also contributed a chapter in Healthcare Technology Solutions for Pandemics – A Roadmap, to be published by Springer Nature.

“Our vision is to provide a machine learning detection framework that can provide early detection for anyone and anywhere,” explains Mohsenin. “The globally collected data from such a framework can be used to study the spread of COVID-19 as well as other viral respiratory diseases among different populations and locations, and to inform and educate the population about how these diseases spread in real time.”

Sajadi says that this COVID-Matter technology will be helpful to him in his practice of medicine. “Being an infectious diseases physician, since the start of the pandemic I have fielded many questions from COVID-19 patients from around the world,” he explains. “When I am talking to a patient, one of my main concerns is determining when they should seek more intensive medical help such as a trip to the ER. A tool such as the one we are developing would be extremely helpful in these situations.”

Using AI to analyze symptoms

The technology will be accessible on smartphones and tablets, which will make it easier for patients to send real-time symptom updates to their physicians. By applying artificial intelligence to analyze symptoms and determine disease severity, physicians can determine if the patient’s symptoms need to be assessed in-person or if virtual care is more appropriate. This enables patients to avoid unnecessary doctor’s office visits, and could potentially help free up medical resources without negatively impacting patient care.

Diagram with boxes indicating clinical symptoms, information available from medical records, and information from an audio spectrogram, and other factors, connected with lines and arrows.
A diagram of the COVID-Matter technology. Image courtesy of Mohsenin.

“Our goal in this research is to allow machine learning models running on general computing processors, such as those found in smartphones and tablets, to assess patients in a way similar to what doctors do at triage and telemedicine,” says Mohsenin. “We want to use passively recorded audio and video and self-declared information to bring proactive healthcare to users’ fingertips. This tool will help users to estimate the urgency and necessity of whether they need to be further examined at a clinic or a specialized facility.”

Making this technology available independent of a medical facility “is critical for early assessment of respiratory symptoms,” she explains, which could help limit future disease spread.

Improving public health

The research team also includes researchers from the University of Maryland School of Medicine and received funding through the Accelerated Translational Incubator Pilot (ATIP) Program. Through a grant competition hosted by the UMB Institute for Clinical and Translational Research, the ATIP award provides seed funding for researchers at UMB and in the community to address issues related to COVID-19 and to improve public health.

ATIP brings together faculty at UMBC and UMB for high-impact multidisciplinary research collaborations. “This is the first time I am working with engineers and it has been an eye-opening experience, from learning the vocabulary to understanding the tremendous potential of AI-based projects. It is something I have enjoyed very much,” says Sajadi.

“With the inputs that we receive from Dr. Sajadi we have been able to improve our machine learning models and assess our overall goals,” says Mohsenin. “Together, we have made significant progress.”

Banner image: Tinoosh Mohsenin. Photo by Marlayna Demond ’11 for UMBC.