Radiologist Work Estimate


Purpose

Automated Computation of Radiologist Time and Effort

Tag(s)

Non-Interpretative

Panel

Business Facing Subpanel

Define-AI ID

19100004

Originator

Summer Kaplan

Lead

Summer Kaplan

Panel Chair

Rich Heller

Non-Interpretive Panel Chairs

Alexander J. Towbin, Adam Prater

Panel Reviewers

Business Facing Subpanel

License

Creative Commons 4.0

Status

Public Comment

Clinical Implementation


Value Proposition


Radiologist interpretive work is reimbursed according to the relative value units (RVUs) assigned to different exams. RVUs are partly based on survey data, which estimate the work involved with exams. These surveys rely on recall of work burden by members of professional societies. In the current system, radiologists are asked to volunteer their time to recall their personal work burden for a given exam by comparing the time, mental effort, judgment, physical effort, technical skill, and the psychological stress of adverse outcomes for the exam in question, compared to other exams. The comparison exams may be similar or completely unrelated, and may or may not be in evaluating radiologist’s scope of practice. This method of valuation is susceptible to recall bias and inaccuracies as well as selection bias in the radiologists who volunteer to participate. Automating real-time assessment for some of these descriptors of radiologists’ work burden could provide a more accurate assessment of physician work without adding to radiologists’ tasks.

Narrative(s)

As a radiologist performs his or her daily work, AI is running in the background, collecting data on radiologist time spent on the case as well as mental effort and stress. Radiologist time spent can be collected from timestamps in radiology information systems or the electronic health record. Data may include those categories defined by American Medical Association/American College of Radiology’s physician work surveys, including pre-service time (work performed before start of exam - may include review of records and any discussions with other physicians or the clinical staff), intra-service time (from time exam begins until it ends - may include QA of images, interpreting the exam, communications with clinicians, review of prior images, image processing, report generation), and post-service tim (activities following report sign off - may include discussions with patient or clinicians, review with other radiologists). The mental effort may be estimated by the length of the report compared with other reports by the same radiologist, number of times exam opened by any radiologist, which may reflect how often radiologist needed to come back to the case and how many other radiologists were asked to look at the case. The mental effort may also be estimated by how many other non-radiologists looked at the images and how many times they were viewed, which may reflect clinical complexity or fragility. Psychological stress may be estimated by the same indicators as mental stress, but weighted by other data such as indication for exam, comorbidities, actuarial data for morbidity and mortality of known diagnoses, and frequency of legal action related to radiology exam code and patient diagnosis codes.

Workflow Description

Work value assessment for selected CPT code is performed. Every 6 months, EHR collects data from multiple sites regarding practice type, radiologist specialty, preservice time, intra-service time, and post-service time, ICD-10 components associated with exam, frequency of stat order, number of times exam opened, frequency of this CPT code and diagnosis-associated ICD-10 codes involved in medical-legal cases, number of images required to review. On an ongoing basis, EHR generates a pop-up survey asking the radiologist to assess psychological stress, mental stress, and physical stress. The survey should be random and infrequent to avoid survey fatigue. Weighted work value is calculated based on these inputs.

Considerations for Dataset Development



Data are CPT code-focused. No initial training needed, start with data collection and ongoing adjustment to output based on new data. Service-related time, frequency of stat order, and the number of times exam opened by others can be drawn from the PACS or electronic health record database for ICD-10 codes associated with CPT? Items not currently in the EHR include actuarial data for ICD-10 codes, association of CPT code and medical-legal cases. Likert survey. How easy to acquire the number of images in the exam? The weighting system for inputs would need to be devised.


Additional considerations include problems that may occur with EHR upgrades or downtimes, links to data collection would need constant maintenance. Inaccuracies could be magnified. Data may need to be handled differently in teaching institutions working with trainees. These inputs do not describe well the work involved in procedures.

Technical Specifications


Inputs

Time rad(s) spent viewing patient's EHR prior to exam

Definition

Total time radiologist(s) spent reviewing patient’s electronic health record before exam


Time since rad opened exam until dictation complete

Definition

Time elapsed between opening exam to complete dictation


Number of times exam opened

Definition

Number of times exam opened


Time spent viewing the exam and EHR after dictation complete

Definition

Total time spent reviewing the exam and electronic health record after dictation complete

ICD-10 codes associated with given CPT

Definition

Mapping between ICD-10 and CPT code


Actuarial risk data for ICD-10

Definition

Actuarial risk data for ICD-10


Frequency of stat orders for CPT

Definition

Frequency of stat orders for the CPT code

Medicolegal risk data for CPT and ICD-10’s

Definition

Medicolegal risk data for CPT and ICD-10 codes

Number of images in the exam

Definition

Number of images in the exam

Practice type

Definition

Practice type

Radiologist specialty

Definition

Subspecialty of the reading radiologist


Primary Outputs


Radiologist Work Estimate

Definition

Radiologist work estimate



Secondary Outputs


Pre-service time

Definition

pre-service time


Intra-service time

Definition

intra-service time


Post-service time

Definition

post-service time


 

Future Development Ideas


  • New exams could be benchmarked against similar existing exams. Exams with a wide range of work value estimates could be studied to determine if there are different groups within to larger group that could be assigned different values

  • Beyond Medicare valuations, practices or departments may wish to use this system to manage their own operations and perhaps weight productivity in their own way. 

  • Different parameters would be needed to describe work involved in procedures.

  • Explore metrics to gauge mental, physical, psychological stressors as they influence relative value scale surveys