ACR Data Science Institute Update: What’s Ahead in 2018
As we begin the new year, the ACR Data Science Institute (DSI) is celebrating its nine-month anniversary. While it might seem odd to count anniversaries in months, the speed of the advances in data science makes just a few months seem like years. At the 2017 RSNA meeting in November, the enthusiasm for learning more about artificial intelligence (AI) in radiological care was palpable, and the ACR DSI was well represented in the RSNA Machine Learning Showcase to share our work with RSNA attendees and industry leaders.
Our numerous meetings with developers and government officials have been extremely positive, indicating that our strategies around how to best leverage the value of radiology professionals to advance the power of AI for medical imaging are being well received in the data science community. As the new year unfolds, the ACR DSI is ready to begin detailing and rolling out specific DSI tools and services to assist developers and government regulators in bringing clinically relevant AI algorithms to clinical practice.
What’s Ahead for the ACR DSI in 2018
The most important activity of the ACR DSI in 2018 will be the development of detailed AI use cases for the radiological sciences. The ACR DSI Data Science Subspecialty Panels are now specifying detailed use cases for AI algorithms that will not only provide a robust narrative description for what a specific algorithm needs to accomplish, but also the mechanisms for training, testing, validation, and monitoring algorithms in clinical practice to ensure they are safe and clinically effective for our patients.
These panels include clinical specialists with expertise in data science, and some use cases will come directly from these panels. However, the DSI will also institute a call for proposed use cases from the entire radiology community, including radiology subspecialty societies, academic and community radiology practices, and developers. These radiology community proposals will be evaluated and prioritized by the DSI Data Science Subspecialty Panels.
Collaborating with Partners
Use case development provides opportunities for the ACR DSI to collaborate within our specialty and, potentially, with other medical specialties. We have already begun collaborative use case development with the Society of Abdominal Radiology, and we believe these types of collaborations will significantly enhance the use case development process. We also look forward to working with academic institutions and developers to garner broad input into use case development.
Finally, collaborations with international organizations to foster consensus for how AI should be regulated globally and how AI can be leveraged to promote better care in resource-challenged areas of the world will be an ongoing ACR DSI activity. International collaborations will allow radiologists to encourage global standards for interoperability and data elements. Understanding global needs can help ensure AI use cases are globally relevant and free of unintended bias.
Tools for Evaluation
Detailed use cases will begin to be available to developers beginning in the second quarter of 2018. Over the course of 2018, one or two ACR DSI use cases will become an end-to-end demonstration of how the DSI use cases and tools will work in practice. The demonstrations will guide the development of specific tools for the FDA to use in its pre-market review and post-market surveillance processes.
We have had a number of discussions around specific novel FDA mechanisms for review of medical devices. The FDA's Medical Device Development Tools (MDDT) program is a way for the FDA to qualify tools that medical device sponsors can use in the development and evaluation of medical devices for pre-market, and many of the DSI tools might qualify as an MDDT tool as well.
The FDA is also creating the National Evaluation System for Health Technology (NEST) to better evaluate medical devices using real-world evidence over the entire lifecycle of the device. The ACR DSI use cases will have data elements designed to populate our new ACR DSI AI Registry, which can show the effectiveness of the algorithms in clinical practice. We have been working with the FDA to promote the ACR DSI Registry as a NEST tool. Recently, our DSI use case, “Lung-RADS Assist: Advanced Radiology Guidance, Reporting, and Monitoring,” was approved by the FDA as a NEST demonstration project.
Establishing these tools for facilitating the FDA review process will allow the DSI to better collaborate with AI developers to create AI algorithms around the DSI use cases. Look for the results of our efforts to qualify a variety of DSI tools in the MDDT program and for the results of the NEST lung cancer screening demonstration project in the coming year.
The ACR DSI is also working with other organizations, including the Medical Image Computing and Computer Assisted Intervention Society, to create model development challenges around the use cases. Challenge participants will be able to have their algorithms ready for the FDA pre-market review process and clinical integration.
Throughout 2018, the ACR DSI will develop a number of educational activities to inform ACR members of key issues in data science. ACR DSI Senior Scientists will contribute articles to the new bimonthly JACR Data Science and Radiological Practice column, with topics including the economics of AI in medical practice, legal and ethical issues around AI in medicine, and government regulatory activities. A number of web-based educational programs are being developed to introduce AI to radiological professionals, detail the implementation of AI, and provide tutorials around the science of AI model creation.
The ACR DSI will also host a number of in-person educational activities. The second ACR DSI Data Science Summit for developers will be held in the late Spring and will focus on the economics of AI development for health care. The ACR DSI will host another in-person data science meeting in the fall of 2018 in conjunction with the annual ACR Quality and Safety Conference, and is partnering with the National Institute of Health and the Radiological Society of North America to host an AI workshop in the summer of 2018.
The coming year promises to be an exciting time for the ACR DSI. Please follow our activities at acrdsi.org.
By Bibb Allen Jr., Chief Medical Officer, ACR Data Science Institute