Table 1 RADAR definition, illustration, and connection to the relevant study designs

1. Technical efficacy

Assessment of the extent to which the AI system can reliably handle tasks and/or process data to produce consistent and robust results

The ability of an AI system to successfully process, analyze, and perform the relevant task on radiological images

• Cross-sectional study to evaluate AI processing speed and image handling

• In silico clinical trial to evaluate AI processing speed and image handling in a simulated setting

2. Diagnostic accuracy efficacy

Assessment of the capability of the AI system to identify, categorize, segment, and/or interpret findings with precision, ensuring that the outputs are useful for subsequent clinical decision-making

The ability of an AI system to correctly diagnose pathologies (e.g., in terms of sensitivity and specificity)

• Cross-sectional study to evaluate AI accuracy on a validation set of images

• In silico clinical trial to evaluate AI accuracy on a validation set of images in a simulated setting

3. Diagnostic thinking efficacy

Assessment of the impact of the AI system’s diagnostic outputs on the radiologist’s or clinician’s diagnostic thought process. The AI system should aid in reducing diagnostic uncertainty and/or enhance the diagnostic decision-making process

The capability of an AI system to optimize the radiologist’s diagnostic process (e.g., by taking away uncertainty in difficult diagnoses)

• In silico clinical trial to evaluate the AI system’s influences radiologists’ behavior in a simulated setting

• Randomized controlled trial to evaluate the AI system’s influences radiologists’ behavior in a controlled setting

4. Therapeutic efficacy

Assessment of the AI system’s influence on the choice and planning of therapeutic interventions. The system should provide actionable insights that guide the clinical management pathway, ensuring the most optimal therapeutic decisions are made

The capacity of an AI system to influence and augment therapeutic decisions (e.g., by resulting in an increase in the number of surgeries performed)

• Randomized controlled trial to evaluate the quality of AI-driven therapeutic decisions in a controlled setting

5. Patient outcome efficacy

Assessment of the direct impact of utilizing the AI system in terms of patient health outcomes, driving improved patient care to ensure holistic well-being for the patient

The ability of an AI system to influence patient outcomes (e.g., in terms of life-years gained, length of stay, patient well-being)

• Randomized controlled trial to evaluate the AI system’s impact by comparing patient outcomes across treatment and exposure arms (e.g., in terms of life-years or QALYs)

• Cohort study to evaluate the AI system’s impact by comparing exposed and non-exposed groups on patient outcomes

6. Cost-effectiveness efficacy

Assessment of cost-effectiveness, weighing both financial implications and societal benefits. The AI system should not only demonstrate value in terms of monetary savings or returns but also improve health at the societal level

The capacity of an AI system to optimize outcomes while minimizing societal costs, evaluated by contrasting long-term benefits (e.g., QALY’s gained) and incremental costs of AI adoption

• Health economic evaluation, such as cost-utility analysis (CUA), to evaluate AI system’s cost-effectiveness

7. Local efficacy

The adaptability of the AI system to the local hospital environment. Evaluates how previously established evidence generalizes to the institutional level due to variations in local circumstances

The extent to which an AI system’s efficacy generalizes to the unique hospital settings (e.g., in terms of workflow, infrastructure, patient demographics)

• Budget impact analysis to assess the impact on the local budget of adopting the AI system

• Multi-criteria decision analysis to enable local decision-makers to consider diverse criteria for informed AI adoption

• Prospective monitoring to ensure long-term local efficacy