Assess-AI provides monitoring of algorithm performance in clinical practice by capturing real-world data during clinical use in a clinical data registry. By providing developers with longitudinal algorithm performance data, we provide a pathway for meeting any FDA post-market surveillance requirements and specific information developers can use for algorithm improvement.

Verifying that an algorithm works well in a clinical setting presents special challenges for developers. The Assess-AI algorithm monitoring service combines real-world data collection of radiologists’ assessments and examination metadata through existing ACR tools (such as TRIAD and DART) and input into proven clinical data registries to provide developers and clinical sites with regular reports of real-world data on an algorithm’s performance across multiple sites

Algorithm monitoring

Using data collected in the ACR National Radiology Data Registry, Assess-AI combines specific information related to an algorithm’s effectiveness reported by radiologists at the point of care, as well as specific metadata related to the exam as specified in the ACR DSI use case.

Metadata includes information such as equipment manufacturer, magnetic field strength or number of CT detectors and other relevant examination parameters, as well as relevant demographic data about the patient. Together these provide Assess-AI with a unique pathway for measuring algorithm performance in real-world settings.

Performance reports

Performance reports are available for both developers and clinical sites using the algorithm. These assure clinical sites that the algorithms are performing as expected and enable developers to make improvements to the algorithms in subsequent versions.

The DSI is currently working closely with the FDA to align Assess-AI with FDA review process criteria in order to provide a pathway for developers to expedite FDA clearance of new AI tools.

If your organization is interested in Assess-AI monitoring services, please contact us.