The Executive Suite: Prominent academic institutions launching Healthcare AI Challenge

A diverse collection of academic health care institutions, and their respective health care professionals, are launching The Healthcare Artificial Intelligence (AI) Challenge, hosted by Mass General Brigham AI. The Healthcare AI Challenge is a virtual, interactive series of events which health care professionals can explore to assess the latest AI health care technologies in real-world scenarios, including features like medical image interpretation, in simulated environments. Health care professionals with relevant credentials can then provide feedback on the performance and utility of the technology, which will generate publicly available insights and analytics.

“The velocity of AI innovations and breadth of their health care applications continues to increase. This unprecedented growth leaves clinicians struggling to determine the effectiveness of these innovations in safely delivering value to health care providers and our patients,” Keith Dreyer, DO, PhD, chief data science officer at Mass General Brigham and leader of Mass General Brigham AI, said in a university release.

“The Healthcare AI Challenge is a collective response to the complexities involved in advancing the responsible development and use of AI in health care. This new approach strives to put clinicians in the driver’s seat, allowing them to evaluate the utility of different AI technologies and ultimately, determine which solutions have the greatest promise to advance patient care.”

Participating and collaborating institutions include Mass General Brigham, Emory Healthcare, the Department of Radiology at the University of Wisconsin School of Medicine and Public Health and the Department of Radiology at the University of Washington School of Medicine, in addition to their associated health care professionals. The American College of Radiology (ACR) also joined the collaborative to ensure access to the challenge for its 42,000-member community.

According to the Mass General Brigham release, the collaborative intends to foster continuous, consistent and reliable expert evaluations of health care AI by crowdsourcing input from credentialed health care professionals. “We need to go beyond collaboratives that come to consensus on how to think about AI,” explained Alistair Erskine, MD, chief information and digital officer at Emory Healthcare and Emory University. “We need health care delivery communities to provide real-world experience of the application of AI at the point of care. That is what the Healthcare AI Challenge is designed to do.”

Contributing institutions collaboratively agreed to begin the challenge with radiology-based AI solutions, given the historic use of AI in the field. “Medical imaging provides many types of data, and up to 95% of health care data is unstructured, non-text data,” said Richard Bruce, MD, associate professor of radiology and vice chair of informatics at the University of Wisconsin School of Medicine and Public Health. “AI has the potential to interpret and distill that data at a new scale and speed, but what we need is the ability to quickly test and compare different AI solutions. The Healthcare AI Challenge will offer a platform to evaluate and compare tools across various clinical situations.”

Health care professionals at collaborating institutions can test any of the foundation models across a series of challenges before rating their clinical skill level. The scores will contribute to the insights and analytics rankings, which can serve to provide stakeholders and the public with a transparent analysis of AI solutions’ performance across various practical scenarios. Feedback can be considered by the public and can provide valuable insight to developers, who can enhance the technologies to ensure that they are fit-for-purpose, commercially viable and clinically relevant.

According to Mass General Brigham, The Healthcare AI Challenge will continue to innovate, adding new AI solutions to its interactive environment. Results from the challenge will be publicly available here.