A Checklist for Radiology Departments Seeking to Purchase AI

Development of artificial intelligence (AI) has led to more widespread use in healthcare, including radiology. There is great hope for AI as a strong companion for radiologists to enhance accuracy, productivity, and workflow by taking over routine tasks and freeing up more time during the workday. Since AI has the potential to impact and improve patient care, many radiology departments are now starting to consider purchasing AI systems. When seeking to invest in AI, radiology departments should do so carefully, similarly to when investing in other health technology solutions.

A trio of imaging informatics experts looked at everything radiology departments need to consider before purchasing AI. Marc Kohli, MD, associate professor and associate chair of clinical informatics and John Mongan, MD, PhD, associate professor and associate chair for translational informatics in the UC San Francisco Department of Radiology and Biomedical Imaging were authors on this article along with Ross Filice, MD, chief of imaging informatics and body imaging at MedStar Georgetown University Hospital. The article was recently published in the Journal of the American College of Radiology.  

“Purchasing an AI solution simply because AI is a hot topic, or because one generally thinks AI will improve the practice of radiology, is not recommended because it may consume resources without benefit to clinical practice,” say the team of scientists. “If a tangible problem is identified and AI tools targeting that problem are commercially available, then one can begin to evaluate whether purchasing an AI tool makes sense.”

As such, they went on to outline a stepwise process for evaluating AI tool purchases, beginning with performance evaluation. A few “take home” points include:

• AI implementations should address a well-defined problem in the radiology practice.
• Ease of use and workflow integration quality should be assessed before and after implementation.
• AI models should be monitored for patient safety, including unintended bias and especially the potential for reinforcing health care disparities.
• Impact on IT infrastructure and cost should be included in return-on-investment calculations.

In addition to their extensive guidelines, the team of experts put together an AI purchasing checklist, shown in the figure below.

“Purchasing AI systems requires close coordination with many stakeholder groups and consideration of system performance, validation, IT requirements, cost, as well as quality and safety,” the team concludes.

Drs. Mongan and Kohli are both part of the leadership of the UCSF Center for Intelligent Imaging (ci2), founded in 2019 to accelerate the application of AI to radiology, leveraging advanced computational techniques and industry collaborations to improve patient diagnoses and care.