Researchers from the Mayo Clinic have developed an AI system, specifically a vision transformer, that can detect surgical site infections (SSIs) from photos submitted by patients. The researchers published their study in the Annals of Surgery.
from Adobe Stock
The team identified a growing workload for clinicians as outpatient surgeries and remote monitoring increase. Clinicians currently manually check patient-submitted wound images via online portals for SSI and other issues. Early identification of SSI is vital for reducing morbidity, but the increasing workload presents a barrier to this need.
Using photos submitted by patients through online portals, the AI system can automatically identify surgical incisions, assess image quality and flag signs of infection. The system was trained on over 20,000 images from over 6,000 patients from nine Mayo Clinic hospitals. The model detects surgical incisions with 94% accuracy and an 81% area under the curve in identifying infections.
“The AI model is based on deep learning, which uses layers of artificial neurons for abstract representation of input images. These layers extract features from patient-submitted images – such as edges and patterns – which may or may not be visible to the human eye but have information that can distinguish images based on different targeted outcomes,” said Cornelius Thiels, D.O., co-senior author of the study. He added that detecting infection is just one part of the system. It can also categorize images and identify low-quality images.
This technology could help reduce delays in diagnosing infections and provide better care for patients recovering from surgery at home. It could allow for earlier treatment of infections, decreased morbidity and reduced costs.
“This work lays the foundation for AI-assisted postoperative wound care, which can transform how postoperative patients are monitored,” says Hala Muaddi, M.D., Ph.D., first author of the study. “For patients, this could mean faster reassurance or earlier identification of a problem,” she added, “For clinicians, it offers a way to prioritize attention to cases that need it most, especially in rural or resource-limited settings.”
The model could have other healthcare applications, said Thiels, “For example, patients submit many types of pictures to their physicians after surgery, including pictures of drains and ostomies. There is an opportunity to leverage similar models and pipelines to automate and improve care for patients while they are in the hospital.”
AI bias
In addition to statistical bias (related to underfitting), AI models are known to contain racial bias. Systems are often trained on light-skinned individuals, meaning they struggle to identify darker skin. Additionally, models have a gender bias, performing better when asked to identify male faces than female faces. In models like this one that are asked to evaluate pictures of patients, decreasing bias is a vital consideration.
“Accounting for bias was an important component of this work. In the initial paper, we reported a sensitivity analysis that stratified by race, which demonstrated comparable performance, suggesting minimal bias and strong generalizability. In addition, we are currently performing additional analysis by skin tone to make sure that the model accurately performs across all patients,” said Thiels.
On SSIs and neural nets
The CDC healthcare-associated infection prevalence survey found that there were an estimated 110,800 SSIs in 2015. SSIs account for 20% of healthcare-associated infections (HAIs). SSI is also the most costly type of HAI, with an estimated annual cost of $3.3 billion. SSIs extend hospital stays by 9.7 days, with the cost of hospitalization increasing by more than $20,000 per admission.
The vision transformer used in this research reflects a growing interest in transformers, the platform coined in the 2017 paper “Attention Is All You Need” that paved the way for smarter, more contextual capabilities in Google search queries and, eventually, ChatGPT and the current genAI wave. Here, vision transformers, also a neural network subtype, offer an alternative to convolutional neural networks, which had been a staple of modern image processing, including AlexNet, the platform Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton introduced in 2012.
“Even if the model only helps one patient, we see this as a win, but we think this has the opportunity to help many patients undergoing surgery at Mayo Clinic and beyond. It is important to note that many tools have the potential to improve outcomes for patients, but one unique aspect of these AI-based models is that there is an opportunity to reduce workload for clinicians and improve access to care at the same time, which is also very important,” said Thiels.