Empowering radiologists to develop AI algorithms at their own institutions, using their own patient data, to meet their own clinical needs, could potentially accelerate the development of AI for diagnostic radiology. Currently, developments in AI for the radiological sciences are being driven by exciting research happening primarily at institutions with extensive informatics and data science resources — and primarily using single institution patient data. While some of these algorithms are achieving performance levels similar to the performance of diagnostic radiologists, it is not clear if and how these tools will be generalized to widespread clinical use across multiple institutions. Without that transition, AI will fail to have broad impact on patient care.

Scaling AI development to meet the clinical need

Not only are most algorithms only being used at the institutions where they were developed, at present there are no clear mechanisms to validate these algorithms across multiple institutions or to provide for clinical integration of AI. Many of the algorithms developed thus far have proven brittle in actual clinical use when tried at different institutions. While we all recognize AI algorithms will be more effective when trained across a wide and diverse array of clinical data, patient privacy concerns are an ongoing issue. Such concerns have prevented the release of large volumes of patient data for use in AI research, AI challenges or commercial development, which has in turn limited AI developers’ access to diverse, well-annotated patient datasets and has constrained the ability of institutions to collaborate on AI development.

Thus far, radiology professionals, who have the requisite domain expertise to make AI relevant for clinical use, have not been able to widely participate in AI development widely because of limited access to AI computational solutions and the complexity of AI computational architecture. The vast majority of us — who do not have the data science training to master AI programming — are essentially unable to participate in AI.

Accelerating radiologists’ contributions to AI development

To maximize the development and adoption of AI in clinical practice, radiologists need to be empowered to create AI tools in their own institutions. Trainees and young professionals in particular need local, hands-on opportunities to learn the basics of developing AI algorithms, as well as access to the computing and logistical solutions needed to develop AI tools that will improve their practices. Providing tools to aggregate and annotate cases — and to run and evaluate AI algorithms, collaborate with developers and other institutions on training AI models, and create their own AI algorithms without patient information ever leaving the facility — offers a potential solution for privacy protection while opening up more data for algorithm training and testing.

What do radiologists need?

The Data Science Institute® (DSI) is launching ACR AI-LAB™, a data science toolkit designed to help radiologists learn the basics of AI and then use these skills in their practices. A series of modules will allow radiologists to first explore using basic AI tools applied to a variety of clinical settings. These narrative step-by-step-videos will present general concepts for imaging AI with hands-on interactive tutorials — with the goal of enabling radiologists to consume, create, collaborate, contribute to, and champion AI in their practices.

In May, attendees at the 2019 ACR Annual Meeting will have the opportunity to use ACR AI-LAB™ cloud-based data and computing tools to annotate images for use in developing AI algorithms. They will be able to explore and experiment with the AI tools necessary to modify and refine AI models. Soon after, the ACR will provide online access to ACR AI-LAB™ and sample data from publicly available patient datasets along with user friendly computational tools that will allow all ACR members opportunities to learn about annotating datasets and training AI models as well as sample the AI tools that can be used to train and modify existing AI algorithms. With these tools, the DSI will be able to set up a number of challenges where radiologists, especially our residents, can begin learning and using basic AI skills and techniques. As more modules are developed, tools for dataset curation and annotation will provide opportunities for practices to create structured datasets around a specific AI use case. Open source AI computational solutions and freely available “base” algorithms will provide institutions the resources needed to improve existing models, develop their own models, or compete in AI challenges around structured DSI use cases using their own patient data. A standard set of APIs will ensure locally developed algorithms can be integrated into reporting platforms, PACS, or EHRs.

The future: bring the computing to the data

ACR TRIAD (Transfer of Images and Data) already connects thousands of radiological practices for the ACR research, accreditation and registry programs. Using this technology, tools for cross-site training of AI algorithms can be provided in a manner that respects the unique characteristics of local populations, imaging equipment and protocols, and ensures protected patient data remains at each local site. The ACR DSI is making the framework for the ACR AI-LAB™ freely available to developers and industry and will work with the vendor community so that their tools will be able to work within the ACR AI-LAB™ platform.

By 2020, we expect the routine use of AI in radiological practice will have begun. We will be sharing algorithms for transfer learning between sites and enabling sites to see the impact of improving algorithms using local patient data. We further anticipate that multiple institutions and AI developers will be able to work together to develop algorithms using this platform and techniques that can be broadly generalized to many clinical practices. After undergoing more rigorous validation and regulatory approval, these algorithms could be deployed for commercial applications.

ACR AI-LAB™ will join the DSI’s robust set of tools, including structured AI use cases (Define-AI), an AI validation program (Certify-AI), and an AI performance monitoring program (Assess-AI). There will be more opportunities than ever for radiology professionals in large and small practices alike to contribute to AI development in our specialties. These solutions will allow any radiologist to participate in AI development, either by curating and annotating cases according to structured AI use cases, or by contributing to actual algorithm development.

AI opportunities are coming to you

While developing the entire infrastructure for our new platform will not happen overnight, we are beginning the process now. The first of our hands-on learning initiatives will take place at the 2019 ACR Annual Meeting , where attendees can use the ACR AI-LAB™ to annotate data to better understand how newly added data changes algorithm performance. Users will be able to assess breast density and annotate cases — which will then be used to train an algorithm to perform the same function. As the meeting progresses, and more and more members contribute to cases, the algorithm will become more accurate.

We are excited to have begun developing projects that will help democratize AI for all radiology professionals. We hope all ACR members will get involved in this exciting opportunity to shape radiology technology. Learn how to be a part of this initiative by contacting dsi@acr.org.



By Bibb Allen, Jr., MD, FACR, ACR DSI Chief Medical Officer | Diagnostic Radiologist, Grandview Medical Center, Birmingham, AL