How do you determine if an AI algorithm can perform well across imaging devices, clinical facilities and demographic populations? Certify-AI offers developers of health care AI an array of services which can be used individually or bundled together to test algorithm performance ahead of any regulatory review by the FDA or other governmental agencies.
Algorithms which are tested solely against single-source training data may fail when presented with data from other sources. By combining a clearly defined use case such as those found in the Define-AI Directory, with a well-curated ‘ground truth’ reference standard dataset, Certify-AI independently evaluates algorithm performance to safeguard against non-generalizable results in routine clinical practice.
Certify-AI datasets are created with cases from multiple institutions and include cases that span the range of known sources of variability. Evaluation of the algorithm entails defining the appropriate statistical performance metrics and minimal acceptable criteria for formal statistical testing.
Because Certify-AI offers both rigorous independent testing and guidance on algorithm performance, evaluating algorithms through Certify-AI is a means of demonstrating performance to the FDA. The DSI is working closely with the FDA to ensure that our certification standards are in line with their approval criteria to minimize the time involved in the regulatory process. The FDA’s Medical Device Development Tool program is one example where Certify-AI can impact the regulatory review process.
If your organization is interested in Certify-AI services, please contact us.