Home

Services

We’re working with a variety of stakeholders — including the developer community — to inform development of clinically relevant AI algorithms that will benefit radiology and improve health care.

Providing expert guidance

DSI operations are led by senior scientists, subspecialty panel members and the ACR AI Advisory Group which includes radiology professionals with clinical and data science expertise as well other statistical and data science experts. Our Subspecialty Data Science Panels evaluate use cases, prioritizing those that can yield the greatest clinical and operational value while considering which problems are most amenable to AI solutions.

Defining problems amenable to AI solutions

Our use cases include both a narrative overview and flowcharts and the technical specifications relevant to the clinical problem. They assist developers by detailing critical content pertinent to algorithm development including the value proposition, common data elements that inform annotation specifications for training and testing, and output parameters including radiologist decision support tools.

Performance evaluation

Certify-AI offers developers a set of services to independently evaluate the performance of algorithms ahead of the FDA pre-market review process using sequestered, multi-institution, well-qualified datasets.

Monitoring performance

Assess-AI provides developers with algorithm performance data which can be used for algorithm improvement and meeting FDA post-market surveillance requirements.

Ancillary services

Dataset directory
Finding the data to train AI algorithms is an ongoing challenge. A directory of health care datasets, ready to be used and analyzed, is available here.

ACR Common
By leveraging existing ontologies and coding schemes such as RadLex Playbook, SNOWMED, CPT, and ICD9+, ACR Common provides common language across all ACR products and services. It is beneficial for organizations in making one-to-one comparisons because it is based on a collection of common radiology terms and semantic structures and organized around fundamental and derived axes such as scenario, procedure, and finding. Learn more .

Image anonymization and data labeling
Protecting personal health information and preparing data appropriately for training and testing an algorithm can be difficult and time-consuming, but it is an area in which the ACR has developed significant experience through its clinical trials networks. We offer image anonymization and annotation tools and services to developers so they in turn can work with radiologists in a wide array of clinical settings to prepare images for use in AI development.

Reader studies
At times, developers may build algorithms that have not yet been specified as a Define-AI use case and will need to establish their own reference standard for training, testing or validating an algorithm. We offer well-designed reader studies that rely on radiology experts. Our reader studies will evaluate the use case to establish expert-level reference datasets.

Multicenter clinical trials
There is no stronger test of algorithm performance than a multicenter clinical trial using real-world data. If a developer chooses to go beyond simple algorithm validation, we offer a pathway for setting up an advanced multicenter clinical trial. ACR clinical trials technology includes an informatics core, protocol and study design, and site management and qualification to ensure that the trial is completed on schedule, and the results are accurate, reproducible, and generalized to the population at large.

Learn more about the services DSI offers by contacting us.

Define-AI Directory

See the full list of freely available use cases

See the list