Skip to main content

Measuring appropriateness of diagnostic imaging: a scoping review

Abstract

In radiology, the justification of diagnostic imaging is a key performance indicator. To date, specific recommendations on the measurement of appropriateness in diagnostic imaging are missing. To map the study literature concerning the definition, measures, methods and data used for analyses of appropriateness in research of diagnostic imaging. We conducted a scoping review in Medline, EMBASE, Scopus and the Cochrane Central Register of Controlled Trials. Two independent reviewers undertook screening and data extraction. After screening 6021 records, we included 50 studies. National guidelines (n = 22/50) or American College of Radiology Appropriateness Criteria (n = 23/50) were used to define and rate appropriateness. 22/50 studies did not provide methodological details about the appropriateness assessment. The included studies varied concerning modality, amount of reviewed examinations (88–13,941) and body regions. Computed tomography (27 studies, 27,168 examinations) was the most frequently analyzed modality, followed by magnetic resonance imaging (17 studies, 6559 examinations) and radiography (10 studies, 7095 examinations). Heterogeneous appropriateness rates throughout single studies (0–100%), modalities, and body regions (17–95%) were found. Research on pediatric and outpatient imaging was sparse. Multicentric, methodologically robust and indication-oriented studies would strengthen appropriateness research in diagnostic imaging and help to develop reliable key performance indicators.

Key points

  • National guidelines or the American College of Radiology Appropriateness criteria were used to rate the appropriateness of diagnostic imaging.

  • Evidence on the appropriateness concerning pediatric/adolescent imaging and outpatient settings is sparse.

  • Heterogeneous appropriateness rates were reported throughout all body regions and modalities.

  • Appropriateness across studies is not comparable because of non-standardised and monocentric data acquisition.

  • Methodologically robust and indication-oriented appropriateness research is needed to improve further analyses.

Introduction

For medical services with particular risks of complications, the benefits of a procedure must outweigh the harmful factors. In radiology, appropriateness is the key element in the justification of diagnostic imaging. Both, the American College of Radiology (ACR) and the European Society of Radiology (ESR) define appropriateness of a radiological procedure by the evidence-based advantageousness of the risk–benefit ratio [1, 2].

An indication for a radiological procedure includes two stages. A referring physician provides the medical indication (referral) and the radiologist justifies or denies the provision of the radiological procedure. Appropriateness, as used in this review, refers to the process of justifying a radiological examination through careful consideration of the risks and benefits associated with the procedure. Improper patient selection causes under-, over- or misuse of radiological procedures [3]. Underuse is critical because of the risk of missing important diagnoses and a resulting delay in patient treatment with consecutive later or wrong diagnosis and treatment. Misuse bears the risk of excess radiation exposure and/or double investigation with consecutive resource waste. Overuse means that examinations are performed without therapeutic implications. Overuse may cause unnecessary radiation exposure and/or an overload of referrals which may lead to the delay of other urgent radiological procedures [3,4,5]. Resource allocation is crucial. Especially patients in countries with low density of large-scale equipment encounter longer waiting times [6].

Costs of radiological procedures are one of many variables in a healthcare system and a balanced justification is needed [2, 7, 8]. However, the ESR emphasises clearly that an individual justification must not be influenced by costs but only a favorable risk–benefit ratio [2].

Recent literature focuses on the appropriateness of referrals and certain indications like low back pain [9], which represents the quality of the medical indication given by a referrer [10].

As the ESR stated in 2020, the monitoring of undertaken diagnostic imaging, particularly high-dose studies, could serve as a key performance indicator for auditing radiation protection [11]. Their statement is rather general and specific recommendations on the realization are missing, as well in the Esperanto Guide to Clinical Audit as in other publications [12,13,14]. Therefore, our review focused on analysing the appropriateness of the diagnostic imaging that was performed, with radiologists serving as the gatekeepers responsible for determining whether the imaging is necessary. As part of our review we identified the following key questions:

  1. (i)

    What is the definition of appropriateness in diagnostic radiology in different study settings?

  2. (ii)

    What are the measures and results of appropriateness in diagnostics in different study settings?

  3. (iii)

    Which methods are used to measure appropriateness in radiological diagnostics?

  4. (iv)

    Which data are used to measure appropriateness in radiological diagnostics?

Methods

We conducted a Scoping Review to answer the objectives mentioned above using the updated guideline for Systematic Scoping Reviews [15]. Scoping reviews are indicated if a research field has not yet been systematically reviewed and the topic is complex. This mapping of evidence initially helps to identify entry points and relevant issues for specific evidence syntheses (including systematic reviews) [16, 17]. The results of scoping reviews are usually analyzed by using descriptive statistical methods and can then be visualized and presented by evidence mapping without critical appraisal [15]. For reporting, we applied the PRISMA-ScR Checklist [18].

Inclusion and exclusion criteria based on the population–concept–context (PCC) framework

Based on the pre-defined inclusion and exclusion criteria (Table 1), we published the protocol (Additional file 1: Supplementary material 1) of this scoping review online at the Center for Open Science Framework (OSF). We excluded radiotherapy, screening studies and animal studies. Due to missing details and or research design we excluded commentaries, case reports and conference papers.

Table 1 Inclusion and exclusion criteria based on the PCC (Population–Concept–Context) framework

Search strategy

A systematic literature search (19/07/2021) was conducted in Medline, EMBASE (via OVID), Cochrane Central Register of Controlled Trials and Scopus. The search strategy contained pre-defined keywords, search and MESH terms (Additional file 1: Supplementary material 2). The published protocol in OSF included five key papers [19,20,21,22,23] to validate the search strategy. One of these five key papers was the study of Cristofaro [19], which turned out to analyze radiology requests but not the appropriateness of the actual diagnostic imaging. As our scoping review focuses on studies with actual radiological diagnostic procedures and omits studies about the quality of referrals, we removed this study from screening, diverging from the initial protocol uploaded at OSF. In addition, already published (systematic) reviews as well as the reference lists of the included articles (backward citation tracking) and articles citing these were screened (forward citation tracking via Scopus).

Study selection

After removing duplicates using Endnote V9, the results were screened by two independent reviewers (F.W., S.B.) at title-/abstract and full-text level using Rayyan (https://rayyan.qcri.org/). In the case of diverging ratings of relevance on the full-text level, both reviewers reached a consensus.

Extraction and synthesis of relevant content

One reviewer performed the data extraction and was checked by the other reviewer. The extraction content included study characteristics, methods and appropriateness results. Many guidelines like the ACR appropriateness criteria (AC) differentiate between usual/full appropriateness, maybe/moderate appropriateness and not appropriate [1]. To keep a dichotomous and interpretable synthesis, and to include all results that have already been aggregated, we interpreted results of a maybe/moderate appropriateness as usual/fully appropriate in the overall rating. If the results were already aggregated to dichotomous results by the study authors, we extracted the results as reported by the literature. If information was missing or results were indeterminate, we highlighted these results as indeterminate/not applicable.

Both reviewers extracted the data in a piloted standardized data extraction spreadsheet (Excel) using five included articles to perform possible modifications before the extraction of all included studies. We synthesized the study results according to the different modalities, applied guidelines and body regions. We summarized the descriptive appropriateness results aggregated by modality and body region.

Results

After screening 6,021 records, 101 full-text publications were screened for eligibility (Fig. 1). The most frequent reasons for exclusion at full-text level (Additional file 1: Supplementary material 3) were conference abstracts (n = 26) [24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49] and studies exclusively investigating the appropriateness of requests for diagnostic imaging (n = 13) [19, 50,51,52,53,54,55,56,57,58,59,60,61]. Finally, 50 studies met the predefined inclusion criteria and were included in this scoping review [20,21,22,23, 62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107].

Fig. 1
figure 1

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews Flowchart. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews Flowchart shows the identified and screened records on title-abstract and full-text basis and the number of finally included studies for data extraction

Table 2 describes the detailed characteristics of the included studies summarized below. Most of the studies were conducted in the USA (16/50), Canada (6/50), Australia (3/50) or Western Europe, particularly the United Kingdom (4/50), Italy (4/50), Spain (3/50), Germany (2/50) or Finland (2/50) (Table 2). The oldest study was published in 1994 [90]. The vast majority of studies (39/50) were published within the past ten years [20,21,22,23, 64,65,66,67,68,69,70,71, 74,75,76,77, 79,80,81,82,83, 85, 87,88,89, 91, 92, 94, 96, 98,99,100,101,102,103,104,105,106,107]. Most of the studies were fully (35/40) or partially (5/40) undertaken in inpatient settings and included varying numbers of participants (1st Percentile: 225, Median: 503, 3rd percentile: 1295). The most common modality was computed tomography (32/50), followed by magnetic resonance imaging (19/50), radiography (15/50) and ultrasound (3/50).

Table 2 Characteristics of included studies

Most of the studies referred to the definitions of appropriateness in guidelines (question 1). Especially national guidelines (22/50) or the ACR AC (23/50) were used to define and judge appropriateness. Based on the applied guidelines, most of the studies provided dichotomous or ordinal ratings (e.g., appropriate, may be appropriate or not appropriate) to measure the appropriateness. Some studies did not distinguish between full, moderate, and no appropriateness and summarized the results densely into appropriate/not appropriate (question 2).

Twenty-two out of 50 studies did not provide details about the methodology of the appropriateness ratings. If specified, a single reviewer (14/28) or at least two independent reviewers (12/28) undertook the ratings of appropriateness (question 3).

Five included studies (10%) were interventional studies [83, 87, 97, 106, 107]. The analyzed interventions included guideline implementations [83, 87, 107], radiological consultations [97] for residents and clinical decision support systems [106]. Two guideline implementations [83, 107] and clinical decision support [106] showed significantly improved appropriateness results. Radiological consultation did not reveal significant changes [97].

49 out of 50 studies were cohort studies with a majority of retrospective designs (39/50). Most were undertaken in monocentric (38/50) settings using clinical data (45/50) for most analyses (question 4). For the most part, statistical results were reported descriptively (24/42). Applying Levels of Evidence (Levels of Evidence for Effectiveness) of the Joanna Briggs Institute, the majority of studies (n = 27/50) were rated with evidence level 4a (descriptive design) or 3e (n = 20/50) for uncontrolled studies [108].

A total of 42 studies provided detailed results of single modalities [20, 21, 23, 62,63,64,65,66,67, 69,70,71,72,73,74,75,76,77, 79,80,81,82,83,84,85,86,87,88,89,90, 92, 93, 98,99,100,101,102,103,104,105,106,107].

Seventeen studies encompassed 6559 MRI examinations [21, 63, 64, 67, 69,70,71, 74, 75, 79, 80, 82, 88, 90, 101, 103, 105] and reported an overall appropriateness of 79% (n = 5204/6559). 26 studies rated 26,715 CTs [21, 23, 63,64,65, 67, 72,73,74, 76, 77, 81, 83, 86, 89, 92, 93, 98,99,100,101,102,103,104, 106, 107] with an overall appropriateness of 60% (n = 16,363/27,309). Eleven studies [20, 23, 62, 66, 83,84,85, 87,88,89, 104] reported an appropriateness of 55% in 7729 reviewed radiographs (n = 4271/7729) and two studies [88, 104] reviewed 1535 radiological ultrasounds with an overall appropriateness of 44% (n = 680/1535). Overall, there are no patterns regarding size of study population and appropriateness (Fig. 2). The detailed study results were classified according to modalities and aggregated to body regions (Fig. 3), head/neck, chest, heart/vessels, abdomen, pelvis and musculoskeletal system, including spine and extremities, whole body and miscellaneous. Please refer to Additional file 1: Supplementary material 4 for the detailed study results (question 2).

Fig. 2
figure 2

Bubble plot of study-individual appropriateness and number of reviewed examinations. The Bubble plot shows a matrix of study-individual appropriateness results (y-axis) and the number of reviewed examinations (x-axis). The age-stratified (adult, children) results were separately presented according to the aggregated body regions chest/abdomen/pelvis, chest/breast, general musculoskeletal (MSK)/extremities, head/neck, heart/vessels, other/miscellaneous, spine and whole body. Interventional studies are highlighted with black circles. Four studies reviewing the appropriateness of imaging other/miscellaneous body regions (CT + MRI) [64], heart/vessels CT [86], chest/breast CT [65] and whole body CT [89] were not included into the bubble plot due to missing information on the number of reviewed images. 1: Identical study results [100] of children and adult CT lead to a total overlap of bubbles in head/neck imaging. 2: Similar study results of adult MRI [93, 101] and adult CT [75] lead to a partial overlap of bubbles in head/neck imaging. CT Computed tomography. MRI Magnetic resonance imaging. MSK Musculoskeletal. US Ultrasound. XR Radiography

Fig. 3
figure 3

Appropriateness of MRI, CT, X-ray (XR) and ultrasound (US) in different body regions. The image shows the absolute number of studies and the accumulated number of reviewed examinations separated into different body regions. Based on the accumulated results, overall appropriateness (in %) was calculated presenting the span of study individual reported appropriateness in percent from lowest to highest. *Four studies reviewing the appropriateness of imaging other/miscellaneous body regions (CT + MRI) [64], heart/vessels CT [86], chest/breast CT [65] and whole body CT [89] reported the relative appropriateness without providing absolute numbers for different body regions. CT Computed tomography. MRI Magnet resonance imaging. XR Radiography. US Ultrasound

Head/neck

The appropriateness of head/neck imaging was analyzed by 14 studies [21, 63, 72, 74, 75, 79,80,81, 84, 93, 100, 101, 104, 105] reviewing MRI [21, 63, 74, 75, 79, 80, 101, 105], CT [21, 63, 72, 74, 81, 93, 100, 101, 104] and radiography [84]. They used national guidelines [21, 63, 72, 80, 84, 104, 105], ACR AC [74, 75, 80, 93, 101], EU guidelines [100] or internal/own guidelines [79].

Six studies focussing on MRI in adults [21, 63, 75, 80, 101, 105] reviewed 1694 examinations with 40 [63] to 1000 [105] records per study. 1500 (89%) head/neck MRIs were rated appropriate ranging between 81% (n = 130/161)[75] over 88% (n = 882/1000)[105] to 97% (n = 239/247) [80] study individual appropriateness. Three studies [74, 79, 80] reviewed 315 pediatric MRIs with an overall appropriateness of 92% ranging between 92 [74, 79] and 89% [80].

Nine studies reviewed 12,631 head/neck CTs [21, 63, 72, 74, 81, 93, 100, 101, 104] separable into 675 CTs of adults/unknown age in seven studies [21, 63, 72, 93, 100, 101, 104] and 11,956 pediatric CTs in three studies [74, 81, 100]. The appropriateness of head/neck CT in adults/patients of unknown age was rated with national guidelines [21, 63, 72, 104], EU guidelines [100] or ACR AC [93, 101]. It varied between 44% (n = 26/59) [63] and 98% (n = 63/64) [104], averaging out at an overall appropriateness of 79% (n = 532/675). Concerning pediatric head/neck imaging rated with on ACR AC [74, 81] or EU guidelines [100], the number of reviewed examinations and appropriateness varied between 97% (n = 29/30) [100], 90% (n = 108/120) [74] and 31% (n = 3660/11,806) [81]. Concerning head/neck radiographs, one study reported an appropriateness of 83% (n = 24/29) [84].

Heart/vessels

Six studies about heart and/or vessel imaging [77, 80, 86, 101, 102, 107] were included. They reviewed 35 adult MRIs [80, 101], two pediatric MRIs [80] and 4836 adult CTs [77, 86, 101, 102, 107]. Two adult MRI studies report heterogeneous appropriateness rates of 100% (ACR AC: n = 21/21) [80] on the one and 0% (national GL: n = 0/14) on the other hand [101], leading to an overall appropriateness of 62% (23/37). One study reviewing two pediatric heart/vessel MRIs reported an appropriateness of 100% [80]. For CT, the overall appropriateness according to ACR AC [77, 101] or national guidelines [86, 102, 107] was 84% (n = 4052/4836). The number of reviewed CTs varied between 32 [101], 243 [77], 1984 [102] and 2577 [107]. The study without information on the number of reviewed examinations reported the lowest appropriateness of 27% (n = N/A) [86], and the study with the highest number of examinations reported the highest appropriateness: 93% (n = 2384/2577) [107].

Chest/breast and thoracoabdominal imaging

13 studies analyzed chest/breast or thoracoabdominal imaging comprising 162 MRIs (adult: 153, children: 9) [21, 80, 88, 101], 1580 CTs [21, 23, 63, 65, 76, 93, 101], 3456 radiographs [23, 84, 88, 89, 104] and 1500 ultrasounds [88]. The overall appropriateness based on national guidelines [21, 23, 63, 65, 80, 84, 88, 104], EU guidelines [89] or ACR AC [76, 80, 89, 93, 101] varied between MRI (n = 69/162, 43%), CT (n = 1203/1580, 76%), radiography (n = 2462/3456, 71%) and ultrasound (n = 645/1500, 43%). Referring to individual study data, the numbers of reviewed examinations ranged from 1 [101] to 100 [88] MRIs, 36 [93] to 1005 [76] CTs and 50 [84] to 2350 [88] radiographs. The study-individual appropriateness results for MRI (17–100%), CT (54–88%) and radiographs (47–99%) varied either. Two studies did not provide information on the absolute number of reviewed images, with one study examining CTs (54% appropriate) [65] and the other examining radiography (no information on the number of images or results available) [89].

Thoracoabdominal imaging has been analyzed in two studies reviewing 160 CTs with an overall appropriateness of 81% [21, 63].

Abdomen

Thirteen studies [21, 23, 62, 63, 66, 73, 80, 84, 87, 93, 100, 101, 104] analyzed the appropriate imaging of abdomen using ACR AC [73, 80, 93, 101], EU guidelines [100], or national guidelines [21, 23, 62, 63, 66, 80, 84, 87, 104].

MRI was analyzed in four studies [21, 63, 80, 101] reviewing 241 adult MRIs and 10 pediatric MRIs [80]. The overall appropriateness was found to be 94% for adults and 100% for children. The appropriateness varied between 83 [21] and 100% [80] in adult MRIs.

Six studies [21, 73, 93, 100, 101, 104] reviewed 2591 CT examinations (adult: 2443, children: 148) with an overall appropriateness rate of 89% (n = 2373/2591). These studies reviewed between 30 [100, 104] and 2022 [73] examinations. They reported appropriateness rates between 42 (n = 126/300) [87] and 99% (n = 2008/2022) [73] for adults and 80% (n = 118/148) for children in one study [101] reviewed with ACR [93, 101] or national guidelines [21, 73, 100, 104].

Radiography was analyzed by five (n = 729) [23, 62, 66, 84, 87] studies and ultrasound by one study (n = 35) [104], respectively. Overall, 311/729 (43%) radiographs were rated appropriate. Solely national guidelines were applied for the ratings. The number of reviewed radiographs per study (n = 16 [84]–n = 225 [62]) and results (32 [62]–69% [84]) varied. The 35 radiological ultrasounds were rated 100% appropriate according to national guidelines [104].

Spine

Fourteen studies rated the appropriateness of spinal imaging [21, 63, 70, 80, 83,84,85, 87, 93, 98,99,100, 105, 106] including 1941 MRIs (adults: 1928, pediatric: 13) [21, 63, 70, 80, 105], 1679 CTs [21, 63, 83, 93, 98,99,100, 106], and 851 radiographs [83,84,85, 87]. Spinal MRI showed an overall appropriateness of 63% (n = 1221/1928) in six studies using ACR AC [80], EU guidelines [70] or national guidelines [21, 63, 80, 105]. However, the appropriateness in the studies ranged between 44 (n = 443/1000) [105] and 88% (n = 530/602) [70] in adults, and 100% (n = 13/13) in one study analysing pediatric MRIs [80]. Concerning spinal CTs, eight studies reported an overall appropriateness rate of 77% (n = 1292/1679) with varying results (28% [21]–96% [99]) and population (n = 17 [63]–n = 507 [98]) per study. Concerning radiography, both, population and appropriateness results varied between 0 (n = 0/433) [85], 74 (n = 84/113) [83] and 91 (n = 10/11) [84] leading to an overall appropriateness of 29% (n = 246/851).

Extremities and general musculoskeletal imaging

Seven studies [21, 63, 69, 71, 84, 100, 104] reviewed extremity and general musculoskeletal imaging including 535 MRIs [21, 63, 69, 71], 73 CTs [21, 100], and 102 radiographs [84, 104]. The appropriateness ratings were based on ACR AC [69, 71], EU guidelines [100] or national guidelines [21, 63, 84, 104]. For MRI of extremities, the overall appropriateness rate was 66% (n = 352/535). The four underlying studies varied in sample size (55 [63]–300 [71]) and results (55 [71]–83% [69]). Two studies analysing CTs of extremities reported heterogeneous appropriateness rates of 51% (n = 22/43) [21] and 90% (n = 27/30) [100], resulting in an overall appropriateness of 67% (n = 49/73). The two studies including radiography of extremities, 68% of the radiographs (n = 69/102) were rated as appropriate with 86% (n = 6/7) [104] and 66% (n = 63/95) [84].

Two studies reviewed 194 MRIs (adults: 173, pediatric: 21) [80, 101] and 13 CTs [101] for general musculoskeletal imaging without further differentiation of body regions. Both referred to ACR AC and found appropriateness rates of 82% (n = 141/173) for adult MRIs [80, 101], 81% for pediatric MRIs (n = 17/21) [80] and 61% (n = 8/13) for CTs [101].

Whole body imaging

Four studies [23, 89, 101] analyzed whole body imaging encompassing 281 CTs [23, 101] and 6 radiographs [84]. One study did not provide information about the number of reviewed CTs and reported an appropriateness rate of 45% [89]. The remaining two studies found an appropriateness rate of 52% (n = 113/217) [23] and 62% (n = 60/64) [101]. One study rated the appropriateness of six radiographs with 83% (n = 5/6) [84].

Miscellaneous

Eleven studies did not classify body regions [20, 23, 63, 64, 67, 82,83,84, 90, 92, 103]. For adults, 1430 MRIs, [63, 64, 67, 82, 90, 103] 3031 CTs [23, 63, 64, 67, 92, 103] and 2516 radiographs [20, 23, 84] were reviewed. One study reviewed 40 miscellaneous pediatric radiographs [83]. According to ACR AC [67, 82, 92, 103], national [20, 23, 63, 64, 84], EU [83] or own guidelines [90], 94% of the MRIs (n = 1340/1430), 87% of CTs (2650/3031), and 45% (n = 1141/2516) of the radiographs were deemed appropriate. Among the studies, there was a broad variation in the number of reviewed examinations and resulting appropriateness rates with 6 (n = 3/6, 50%) [63] and 1215 (n = 1154/1215, 95%) [103] for MRIs, 192 (n = 110/192, 57%) [23] to 1870 (n = 1746/1870, 93%) [103] for CTs, and 3 (n = 1/3, 33%) [84] to 1977 (n = 922/1977, 46%) for radiographs [20]. One study that reviewed pediatric radiographs reported an appropriateness of 92% (n = 37/40) [83].

Eight studies [22, 68, 78, 91, 94,95,96,97] reviewed 6303 diagnostic examinations completely missing a specification of single body regions and three studies did not differentiate modalities [94, 95, 97]. Here, the overall appropriateness was 72% (n = 4548/6303). The number of reviewed examinations and resulting appropriateness rates varied between 52 (n = 50/52, 96%) [91] and 3079 (n = 2340/3079, 76%) [95].

One study about the appropriateness of CT examinations (33%) and MRI (N/A) did not provide absolute frequencies of the reviewed examinations and therefore did not enter the overall calculation and Fig. 2, as described above [64].

Discussion

International radiological societies regularly define and update appropriateness criteria in order to improve quality, reduce unnecessary radiation exposure and reduce unnecessary costs [8]. An important step in achieving an efficient delivery of diagnostic imaging is to monitor the appropriateness rates. These rates are determined by the rate of by calculating the proportion of guideline-appropriate diagnostic imaging procedures to the total number of diagnostic imaging procedures performed. To date, no key performance indicators have been defined in this regard, and a robust methodology to derive them is indispensable.

This review presents several important new findings that are relevant to evaluate the appropriateness of radiological imaging in daily practice and research:

  • More than 80% of the included literature relied on national or ACR AC to rate the appropriateness of diagnostic imaging. This emphasizes the importance of specific guidelines that can serve as a tool to rate appropriateness.

  • All included studies reported appropriateness as a percentage. The benefit of a percentage is the applicability as key performance indicator. The studies presented a broad range of included examinations (88–11806) depending on the body region, the patient group and the modality.

  • Many appropriateness ratings were methodically unclear (n = 22/50, 44%). Of the 28 studies reporting the rating methodology, less than 50% employed two independent reviewers. The data indicate that a small number of the included studies used double-independent appropriateness ratings. Double independent reading and justification is a common method in clinical medicine to reduce errors and identify discrepancies. It is also a standard methodological practice in systematic reviews and the coding of interviews in qualitative research [109,110,111,112,113]. Therefore, we assume that the raters who did use double-independent ratings had a higher level of awareness, as such a rating approach requires standardization and transparent a priori definitions [114]. Measurement of appropriateness was mainly based on guidelines but was aggregated completely or by multiple body regions in some studies [21, 103]. This simplification impedes a detailed analysis of concordance with indication-driven guidelines. Although guidelines seem to describe an imaging pathway clearly, one reader is not sufficient. One obstacle is that several studies found contradictory results as soon as 2 guidelines existed for one pathology [76, 84, 98, 99]. In order to compare the appropriateness on a national or even international scale, a harmonization among guidelines of different authorities is demanded. It was also shown that one guideline can lead to different decisions after an update [77, 83]. For this reason, a double-reading approach is needed to objectively evaluate the appropriateness of radiological imaging and the appropriateness must be rated in accordance with the timepoint of every single diagnostic imaging. A homogeneous aggregation of ACR ratings was used throughout this review to facilitate the comparison of different study results, which partially already were aggregated.

  • According to the overall results of this review, 21% of the MRIs, 40% of the CTs, 44% of the radiographs, and 56% of the ultrasound examinations were not appropriate. Thus, thoroughly monitoring appropriateness rates bears high potential for resource management and radiation protection. At the same time, these appropriateness rates are key performance indicators for the gatekeeping function of radiologists.

For the sake of comparability, future studies or national/international audits should either apply a homogeneous aggregation or a separate specification of the appropriateness categories. Our review did not reveal patterns of study characteristics spawning high or low appropriateness rates, which further impedes a comparison between studies.

Assumingly, one reason for the great variety of appropriateness rates is the inhomogeneous application of guidelines in different indications and country/healthcare settings. Moreover, the analyses usually excluded non-codable and uncertain indications [115]. Including non-codable and uncertain indications would strengthen the reliability of appropriateness results. Thus, this knowledge should be exploited and not excluded from studies. The vast majority of studies reviewed inpatient settings, so a lack in the outpatient field must be stated. Importantly, investigation of appropriateness in pediatric populations was sparse, although children and adolescents are more vulnerable to x-ray exposure than adults. All these fields represent important future research topics.

After the creation of guidelines, the real implementation should be monitored. Blachar et al. found a substantial decrease in utilization of MRI and CT after the implementation of the ACR AC and the Royal College of Radiologist guidelines. They also reported a significant decrease in the costs paid by healthcare providers for CT and MRI [116]. More interventional studies could analyze how the implementation of guidelines affects appropriateness rates. Additionally, they would give more insights on the interdependence between the processes of diagnostic imaging, appropriateness, costs, medical treatment and outcomes.

Clinical decision support (CDS) tools are an emerging technique aimed at improving patient safety and promoting value-based imaging [117]. To our knowledge, the influence of a CDC on the appropriateness of radiologists has not been investigated as it was designed for referring providers [117]. According to a large randomized trial, CDS significantly reduced targeted imaging orders by 6%, but did not result in a significant change in the number of high- or low-cost scans. The authors of this study assume that CDS may lead to a modest improvement in the appropriateness of high-cost imaging [118].

Our review also has some limitations. Firstly, as with most scoping reviews, our analysis focused on the big picture with a high degree of aggregation and as a result, we may have missed specific details related to examinations that were rated as "maybe appropriate". Secondly, certain factors that could be associated with appropriateness, such as the quality of referrals, the nature of the disease (acute or chronic), and the type of disease or symptoms were not analyzed.

This review revealed heterogeneities in current evidence concerning study design, statistical methods, reporting and the appropriateness rating itself. This results in challenges regarding the appraisal of study validity due to small sample sizes, conflicting results and lack of reporting. Therefore, the following needs for further research and clinical practice can be derived from this review:

  • An inevitable prerequisite for the measurement of appropriateness rates are evidence-based guidelines. Most studies used clinical data to assess the appropriateness of an imaging procedure. So far, clinical data entail difficulties as they are not standardised. As a consequence, clinical data sets are hard to compare within a single study and especially over several studies. Structured data might be a way forward to gain comparability and transparency. Furthermore, structured data or at least automated data are the very basis for intelligent tools like clinical decision support, which is requested by radiological societies such as the ESR [7, 11].

  • A clear methodological pathway is needed for the measurement of appropriateness rates to obtain more valid data. In this review a high percentage of individual expert opinions were used to define appropriateness. International comparisons of appropriateness rates are impaired by the usage of different national guidelines.

  • Thresholds should be defined while creating certain guidelines to establish awareness for both, the referrer and the radiologist. This could be applied to guidelines for very common indications. To discover applicable numbers, single and specific indications need to be subject of further studies. The design of these studies should include a large number of patients and elaborate specific key performance indicators for different modalities.

Based on this standardization, national audits as well as benchmarking of appropriateness in diagnostic imaging might be feasible. In summary, the following methodological standards should be met in future research or audits about imaging appropriateness to achieve a high level of evidence:

  1. (1)

    Implementation of multicentric studies, preferably with randomized controlled or interventional design;

  2. (2)

    Focus on clinical picture, not body regions, to create an inference to specific guidelines;

  3. (3)

    Consideration of the presence and quality of referrals for diagnostic imaging, as well as the existence of preliminary examinations;

  4. (4)

    Double independent readings of appropriateness;

  5. (5)

    High transparency regarding the rating results;

  6. (6)

    Analytical statistics with clearly defined influencing factors.

Conclusion

In conclusion, many of the reported appropriateness rates might not be representative and cannot be taken as key performance indicators. More reliable and elaborate appropriateness rates with a valid methodical basis are needed. Especially the lack of knowledge about appropriateness rates in pediatric and outpatient imaging should be addressed. This review underlines the need of advanced research concerning appropriateness of clinical care in general and particularly of diagnostic imaging. Appropriateness and quality of indication in general reveal high importance for either medical care, clinical processes and quality.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files. Other data and materials not already published in the protocol, manuscript and supplement will be made available by the authors upon reasonable request.

Abbreviations

ACR:

American College of Radiology

ACR AC:

American College of Radiology Appropriateness Criteria

CT:

Computed tomography

ESR:

European Society of Radiology

MRI:

Magnetic resonance tomography

MSK:

Musculoskeletal

OSF:

Center for open science framework

PCC:

Population–concept–context

US:

Ultrasound

XR:

Radiography

References

  1. American College of Radiology: ACR appropriateness criteria: overview. https://www.acr.org/Clinical-Resources/ACR-Appropriateness-Criteria/Overview (2022). Accessed 21 Jul 2022.

  2. European Society of Radiology (ESR) (2019) Methodology for ESR iGuide content. Insights Imaging 10(1):32. https://doi.org/10.1186/s13244-019-0720-z

    Article  Google Scholar 

  3. Hofmann B, Andersen ER, Kjelle E (2021) Visualizing the invisible: invisible waste in diagnostic imaging. Healthcare. https://doi.org/10.3390/healthcare9121693

    Article  PubMed  PubMed Central  Google Scholar 

  4. Hendee WR, Becker GJ, Borgstede JP et al (2010) Addressing overutilization in medical imaging. Radiology 257(1):240–245. https://doi.org/10.1148/radiol.10100063

    Article  PubMed  Google Scholar 

  5. Rao VM, Levin DC (2012) The overuse of diagnostic imaging and the choosing wisely initiative. Ann Intern Med 157(8):574–576. https://doi.org/10.7326/0003-4819-157-8-201210160-00535

    Article  PubMed  Google Scholar 

  6. OECD. Health at a Glance 2021. 2021.

  7. Brown SD, Hardy SM, Bruno MA (2022) Rationing and disparities in health care: implications for radiology clinical practice guidelines. J Am Coll Radiol 19(1, Part A):84–89. https://doi.org/10.1016/j.jacr.2021.09.023

    Article  PubMed  Google Scholar 

  8. Wintermark M, Rosenkrantz AB, Rezaii PG et al (2021) Predicted cost savings achieved by the radiology support, communication and alignment network from reducing medical imaging overutilization in the medicare population. J Am Coll Radiol 18(5):704–712. https://doi.org/10.1016/j.jacr.2020.12.011

    Article  PubMed  Google Scholar 

  9. Jenkins HJ, Downie AS, Maher CG, Moloney NA, Magnussen JS, Hancock MJ (2018) Imaging for low back pain: is clinical use consistent with guidelines? A systematic review and meta-analysis. Spine J 18(12):2266–2277. https://doi.org/10.1016/j.spinee.2018.05.004

    Article  PubMed  Google Scholar 

  10. Hasenfuß G, Fölsch UR (2017) Klug entscheiden: Rationale, Umsetzung, Evaluation. Z Evid Fortbild Qual Gesundhwes 129:37–40. https://doi.org/10.1016/j.zefq.2017.10.011

    Article  PubMed  Google Scholar 

  11. Mildenberger P, Brady AP, Onur M et al (2020) Performance indicators for radiation protection management: suggestions from the European Society of Radiology. Insights Imaging 11(1):134. https://doi.org/10.1186/s13244-020-00923-1

    Article  Google Scholar 

  12. Howlett DC, Law C, Brady AP et al (2020) The current status of radiological clinical audit and feedback on the ESR guide to clinical audit in radiology and the ESR clinical audit tool (Esperanto) – an ESR survey of European radiology departments. Insights Imaging 11(1):37. https://doi.org/10.1186/s13244-020-00843-0

    Article  Google Scholar 

  13. ESR Esperanto (2019) The ESR guide to clinical audit and clinical audit tool. https://www.myesr.org/media/4136

  14. European Society of Radiology (ESR) (2018) The ESR audit tool (Esperanto): genesis, contents and pilot. Insights Imaging 9(6):899–903. https://doi.org/10.1007/s13244-018-0651-0

    Article  Google Scholar 

  15. Peters MDJ, Marnie C, Tricco AC et al (2020) Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 18(10):2119–2126. https://doi.org/10.11124/jbies-20-00167

    Article  PubMed  Google Scholar 

  16. Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D (2016) An evidence-based approach to scoping reviews. Worldviews Evid Based Nurs 13(2):118–123. https://doi.org/10.1111/wvn.12144

    Article  PubMed  Google Scholar 

  17. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E (2018) Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18(1):143. https://doi.org/10.1186/s12874-018-0611-x

    Article  PubMed  PubMed Central  Google Scholar 

  18. Tricco AC, Lillie E, Zarin W et al (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169(7):467–473. https://doi.org/10.7326/m18-0850

    Article  PubMed  Google Scholar 

  19. Cristofaro M, Busi Rizzi E, Schininà V, Chiappetta D, Angeletti C, Bibbolino C (2012) Appropriateness: analysis of outpatient radiology requests. Radiol Med 117(2):322–332. https://doi.org/10.1007/s11547-011-0725-2

    Article  CAS  PubMed  Google Scholar 

  20. Bertin CL, Ponthus S, Vivekanantham H, Poletti PA, Kherad O, Rutschmann OT (2019) Overuse of plain abdominal radiography in emergency departments: a retrospective cohort study. BMC Health Serv Res. https://doi.org/10.1186/s12913-019-3870-2

    Article  PubMed  PubMed Central  Google Scholar 

  21. Bouëtté A, Karoussou-Schreiner A, Ducou Le Pointe H et al (2019) National audit on the appropriateness of CT and MRI examinations in Luxembourg. Insights Imaging. https://doi.org/10.1186/s13244-019-0731-9

    Article  PubMed  PubMed Central  Google Scholar 

  22. Martins R, Raimundo P, Alves P et al (2020) Appropriateness of radiology test requests by an emergency department: a retrospective study. Acta Med Port 33(1):7–14. https://doi.org/10.20344/amp.12075

    Article  PubMed  Google Scholar 

  23. Vilar-Palop J, Hernandez-Aguado I, Pastor-Valero M, Vilar J, González-Alvarez I, Lumbreras B (2018) Appropriate use of medical imaging in two Spanish public hospitals: a cross-sectional analysis. BMJ Open. https://doi.org/10.1136/bmjopen-2017-019535

    Article  PubMed  PubMed Central  Google Scholar 

  24. Salman L, Edwards Y, Majeed M (2019) Adherence to diagnostic guidelines for suspected pulmonary embolism. Am J Respir Crit Care Med 199(9):A2006

  25. Hextrum S, Ortiz J, Hornik J, Rosenblum J, Bar B (2017) Emergent computed tomography angiography of intra-and extracranial vessels; ordering patterns and yield at a single institution. Ann Neurol 82(Supplement 21):S100. https://doi.org/10.1002/ana.25024

    Article  Google Scholar 

  26. Gorey D, Umana E, Anderson T (2017) Emergency department: are we doing too many chest X-rays. Ir J Med Sci 186(6):S252. https://doi.org/10.1007/s11845-017-1629-5

    Article  CAS  Google Scholar 

  27. Goiney C, Carlson B, Relyea-Chew A, Creutzfeldt C, Yuan C, Mossa-Basha M (2017) Imaging appropriateness criteria may guide effective use of CT angiography in acute stroke workup. Stroke 48(Supplement 1)

  28. Figar S, Findakly S, Wong C, Du L (2017) Retrospective audit of plain film imaging in acute ankle trauma: Are we choosing wisely? J Med Imaging Radiat Oncol 61(Supplement 1):149–150. https://doi.org/10.1111/1754-9485.12657

    Article  Google Scholar 

  29. Ngo D, Leong K, Marks G, Somasundaram A, Richards J, Du L (2016) A single centre audit to assess clinical decision rules for imaging in suspected PE based on RANZCR guidelines. J Med Imaging Radiat Oncol 60(Supplement 1):13. https://doi.org/10.1111/17549485.12519

    Article  Google Scholar 

  30. Kelleher K, Ahmed I (2016) An audit of abdominal x-rays; if their indications are appropriate, in particular for constipation, in a paediatric population. Ir J Med Sci 185(SUPPL. 5):S202–S203. https://doi.org/10.1007/s11845-016-1467-x

    Article  Google Scholar 

  31. Cheng C, Brownlee R (2016) A review of computed tomography imaging requests in the emergency department-Are automated approval systems the answer or part of the problem? J Med Imaging Radiat Oncol 60(Supplement 1):14. https://doi.org/10.1111/17549485.12519

    Article  Google Scholar 

  32. Bonner R, Barnewolt B, Halin N, Mostofi M (2016) Portable chest x-ray utilization in the emergency department. Acad Emerg Med 23(SUPPL. 1):S57. https://doi.org/10.1111/acem.12974

    Article  Google Scholar 

  33. Beinvogl B, McSweeney M, Sabharwal S, Nurko S (2016) Abdominal radiographs: Appropriately used in the management of functional constipation in children? J Pediatr Gastroenterol Nutr 63(Supplement 2):S90. https://doi.org/10.1097/01.mpg.0000503536.79797.66

    Article  Google Scholar 

  34. M Chew CY (2015) Clinical information supplied to radiologists, case note entries, and correlation with subsequent CT findings-A pilot study as part of a quality improvement project. J Med Imaging Radiat Oncol 59(SUPPL. 1):79. https://doi.org/10.1111/1754-9485.12397

    Article  Google Scholar 

  35. Fujimoto J, Karlin D, Ma J (2015) Head imaging in the primary care setting: an analysis of ordering patterns, cost, and outcomes for minor head trauma, headaches, and migraines. J Investig Med 63(1):188. https://doi.org/10.1097/JIM.0000000000000133

    Article  Google Scholar 

  36. Rupasinghe SN, Zikry MS, Marsden MR (2014) Making emergency CT more efficient out of hours: putting the surgeons in charge. Int J Surg 12(SUPPL. 3):S91

    Article  Google Scholar 

  37. Kalsy N, Webb J (2014) Evaluation of the appropriateness of whole-body computer tomography (WBCT) in trauma patients in a tertiary trauma centre. Clin Radiol 69(SUPPL. 1):S23. https://doi.org/10.1016/j.crad.2014.05.097

    Article  Google Scholar 

  38. Yang T, Assaad M, Ananthasubramaniam K (2013) Impact of implementing strategies to improve appropriate use of cardiac computed tomography angiography in clinical practice. J Cardiovasc Comput Tomogr 7(SUPPL. 3):S67–S68

    Google Scholar 

  39. Friedrich GJ, Maeser C, Hommel H, Pachinger O (2013) The application of coronary computer tomography appropriateness criteria in clinical practice: a critical single center evaluation. Eur Heart J 34(SUPPL. 1):845

    Google Scholar 

  40. Tang Y, Parai R, Pandya M, Patel L, Keller S (2012) Are clinicians following the ACG or ACR guidelines when suspecting acute pancreatitis? Am J Gastroenterol 107(SUPPL. 1):S110. https://doi.org/10.1038/ajg.2012.269

    Article  Google Scholar 

  41. Saccoia L, Jones DN, Quinn S et al (2012) Reducing the inappropriate use of medical imaging in the emergency department: a NHRMC TRIP Fellowship Project. J Med Imaging Radiat Oncol 56(SUPPL. 1):4. https://doi.org/10.1111/j.1754-9485.2012.02422.x

    Article  Google Scholar 

  42. Pollentine AA, Edey A, Chandratreya L (2012) Introducing a cardiac CT service - ensuring appropriate referrals and overcoming teething problems. Clin Radiol 67(SUPPL. 1):S18. https://doi.org/10.1016/j.crad.2012.06.093

    Article  Google Scholar 

  43. Carpeggiani C, Morales MA, Marraccini P, Mazzarisi A, Picano E (2012) Inappropriateness of cardiovascular radiological imaging testing in a tertiary care referral center. Eur Heart J 33(SUPPL. 1):247. https://doi.org/10.1093/eurheartj/ehs281

    Article  Google Scholar 

  44. Ajayi OO, Hussain S (2012) Is vetting CTPAs an appropriate use of radiologists’ time? Clin Radiol 67(SUPPL. 1):S2. https://doi.org/10.1016/j.crad.2012.06.003

    Article  Google Scholar 

  45. Sidhu M, Uthamalingam S, Lumish H et al (2011) Adherence to the american college of cardiology foundation/society of cardiac computed tomography - 2010 appropriate use criteria guidelines by cardiologists in an academic tertiary health care institution. J Cardiovasc Comput Tomogr 5(4 SUPPL. 1):S31–S32

    Google Scholar 

  46. Sheikh K, Belfi L, Baad M, Sanelli P (2011) Imaging of acute blunt cervical spine trauma based on ACR appropriateness criteria. Emerg Radiol 18(6):464. https://doi.org/10.1007/s10140-011-0991-2

    Article  Google Scholar 

  47. Enterline D, Rowley H (2011) Evaluation of patients referred for neuroimaging: Which test and why? AJR Am J Roentgenol 196(5 SUPPL.):A226

    Google Scholar 

  48. Sarai P, Sebepos-Rogers G, Mann SD (2010) CT pneumocolon in a district general hospital-is this test being used appropriately? Gut 59(Supplement 1):A121–A122. https://doi.org/10.1136/gut.2009.209007j

    Article  Google Scholar 

  49. Atkinson L, Pratt S, Slevin T et al (2009) Towards appropriate use of diagnostic imaging in general practice. J Med Imaging Radiat Oncol 53(SPEC. ISS. 1):A76. https://doi.org/10.1111/j.1440-1673.2009.01209.x

    Article  Google Scholar 

  50. Lee B, Mafi J, Patel MK et al (2021) Quality improvement time-saving intervention to increase use of a clinical decision support tool to reduce low-value diagnostic imaging in a safety net health system. BMJ Open Qual. https://doi.org/10.1136/bmjoq-2020-001076

    Article  PubMed  PubMed Central  Google Scholar 

  51. Krogh SB, Jensen TS, Rolving N et al (2021) Categorisation of lumbar spine MRI referrals in Denmark as compliant or non-compliant to international imaging guidelines: an inter-rater reliability study. Chiropr Man Ther. https://doi.org/10.1186/s12998-021-00370-9

    Article  Google Scholar 

  52. Bottari G, Stellacci G, Ferorelli D et al (2021) Imaging appropriateness in pediatric radiology during COVID-19 pandemic: a retrospective comparison with No COVID-19 Period. Children. https://doi.org/10.3390/children8060463

    Article  PubMed  PubMed Central  Google Scholar 

  53. Costello JE, Shah LM, Peckham ME, Hutchins TA, Anzai Y (2020) Imaging appropriateness for neck pain. J Am Coll Radiol 17(5):584–589. https://doi.org/10.1016/j.jacr.2019.11.005

    Article  PubMed  Google Scholar 

  54. Poeran J, Mao LJ, Zubizarreta N et al (2019) Effect of clinical decision support on appropriateness of advanced imaging use among physicians-in-training. AJR Am J Roentgenol 212(4):859–866. https://doi.org/10.2214/AJR.18.19931

    Article  PubMed  Google Scholar 

  55. Ranta A, Weatherall M, Gommans J, Tilyard M, Odea D, Dovey S (2017) Appropriateness of general practitioner imaging requests for transient ischaemic attack patients: secondary analysis of a cluster randomised controlled trial. J Prim Health Care 9(2):131–135. https://doi.org/10.1071/HC17005

    Article  PubMed  Google Scholar 

  56. Sodhi KS, Krishna S, Saxena AK, Sinha A, Khandelwal N, Lee EY (2015) Clinical application of “Justification” and “Optimization” principle of ALARA in pediatric CT imaging: ‘How many children can be protected from unnecessary radiation?’ Eur J Radiol 84(9):1752–1757. https://doi.org/10.1016/j.ejrad.2015.05.030

    Article  PubMed  Google Scholar 

  57. van Schouwenburg F, Ackermann C, Pitcher R (2014) An audit of elective outpatient magnetic resonance imaging in a tertiary South African public-sector hospital. S Afr J Radiol. https://doi.org/10.4102/sajr.v18i1.689

    Article  Google Scholar 

  58. Griffith JK, Borycki EM, Kushniruk AW (2014) Diagnostic imaging ordering practices: physician perspectives and implications for decision support. Healthc Q 17(2):62–70. https://doi.org/10.12927/hcq.2014.23876

    Article  PubMed  Google Scholar 

  59. Freeman R, Khanna S, Ricketts D (2013) Inappropriate requests for magnetic resonance scans of the shoulder. Int Orthop 37(11):2181–2184. https://doi.org/10.1007/s00264-013-1968-4

    Article  PubMed  PubMed Central  Google Scholar 

  60. Loughborough W (2012) Development of a plain radiograph requesting algorithm for patients presenting with acute abdominal pain. Quant Imaging Med Surg 2(4):239–244. https://doi.org/10.3978/j.issn.2223-4292.2012.09.06

    Article  PubMed  PubMed Central  Google Scholar 

  61. Nawaz M, Amin A, Qureshi AN, Jehanzeb M (2009) Audit of appropriateness and outcome of computed tomography brain scanning for headaches in paediatric age group. J Ayub Med Coll Abbottabad JAMC 21(1):91–93

    PubMed  Google Scholar 

  62. Morris-Stiff G, Stiff RE, Morris-Stiff H (2006) Abdominal radiograph requesting in the setting of acute abdominal pain: temporal trends and appropriateness of requesting. Ann R Coll Surg Engl 88(3):270–274. https://doi.org/10.1308/003588406X98586

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Lehnert BE, Bree RL (2010) Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: How critical is the need for improved decision support? J Am Coll Radiol 7(3):192–197. https://doi.org/10.1016/j.jacr.2009.11.010

    Article  PubMed  Google Scholar 

  64. Eddy K, Ednie A, Connell C, Eddy R, Eaton K, Mathieson J (2013) Appropriate use of CT and MRI in British Columbia. Br Columbia Med J 55(1):22–25

    Google Scholar 

  65. Sattar A, Khan SA, Al-Qamari N, Adel H, Adil SO, Shafique K (2018) Appropriateness and clinical outcome of chest computed tomography without intravenous contrast: a study conducted in Pakistan. Respir Invest 56(4):342–348. https://doi.org/10.1016/j.resinv.2018.02.005

    Article  Google Scholar 

  66. Fernandez M, Craig S (2019) Appropriateness of adult plain abdominal radiograph requesting in a regional emergency department. J Med Imaging Radiat Oncol 63(2):175–182. https://doi.org/10.1111/1754-9485.12847

    Article  PubMed  Google Scholar 

  67. Fouche PE, Jenkins LS, Vermeulen A (2020) Appropriateness of computed tomography and magnetic resonance imaging scans in a rural regional hospital in South Africa: a 6-year follow-up study. S Afr Med J 111(1):46–51. https://doi.org/10.7196/SAMJ.2020.V111I1.14860

    Article  CAS  PubMed  Google Scholar 

  68. Becker J, Jenkins LS, Swardt MD, Sayed R, Viljoen M (2014) Appropriateness of computed tomography and magnetic resonance imaging scans in the Eden and Central Karoo districts of the Western Cape Province, South Africa. S Afr Med J 104(11):762–765. https://doi.org/10.7196/SAMJ.8158

    Article  CAS  PubMed  Google Scholar 

  69. Glover M, Gottumukkala RV, Sanchez Y et al (2018) Appropriateness of extremity magnetic resonance imaging examinations in an academic emergency department observation unit. West J Emerg Med 19(3):467–473. https://doi.org/10.5811/westjem.2018.3.35463

    Article  PubMed  PubMed Central  Google Scholar 

  70. Kovacs FM, Arana E, Royuela A et al (2013) Appropriateness of lumbar spine magnetic resonance imaging in Spain. Eur J Radiol 82(6):1008–1014. https://doi.org/10.1016/j.ejrad.2013.01.017

    Article  PubMed  Google Scholar 

  71. Gómez-García JM, Gómez-Romero FJ, Arencibia-Jiménez M, Navarro-Gracia JF, Sánchez-Mollá M (2018) Appropriateness of magnetic resonance imaging requested by primary care physicians for patients with knee pain. Int J Qual Health Care 30(7):565–570. https://doi.org/10.1093/intqhc/mzy067

    Article  PubMed  Google Scholar 

  72. Ravindran V, Sennik D, Hughes RA (2007) Appropriateness of out-of-hours CT head scans. Emerg Radiol 13(4):181–185. https://doi.org/10.1007/s10140-006-0531-7

    Article  PubMed  Google Scholar 

  73. Guimarães LS, Fidler JL, Fletcher JG et al (2010) Assessment of appropriateness of indications for CT enterography in younger patients. Inflamm Bowel Dis 16(2):226–232. https://doi.org/10.1002/ibd.21025

    Article  PubMed  Google Scholar 

  74. Rao S, Rao S, Rincon S et al (2016) Assessment of pediatric neurotrauma imaging appropriateness at a level I pediatric trauma center. J Am Coll Radiol 13(7):788–793. https://doi.org/10.1016/j.jacr.2016.02.022

    Article  PubMed  Google Scholar 

  75. Piersson AD, Nunoo G, Gorleku PN (2018) An audit of clinical practice, referral patterns, and appropriateness of clinical indications for brain MRI examinations: a single-centre study in Ghana. Radiography 24(2):e25–e30. https://doi.org/10.1016/j.radi.2017.10.004

    Article  CAS  PubMed  Google Scholar 

  76. Hu-Wang E, Kureshi F, Leifer ES et al (2020) Comparison of professional medical society guidelines for appropriate use of coronary computed tomography angiography. J Cardiovasc Comput Tomogr 14(6):478–482. https://doi.org/10.1016/j.jcct.2020.01.014

    Article  PubMed  Google Scholar 

  77. Mazimba S, Grant N, Parikh A et al (2012) Comparison of the 2006 and 2010 cardiac CT appropriateness criteria in a real-world setting. J Am Coll Radiol 9(9):630–634. https://doi.org/10.1016/j.jacr.2012.04.016

    Article  PubMed  Google Scholar 

  78. De Filippo M, Corsi A, Evaristi L et al (2011) Critical issues in radiology requests and reports. Radiol Med 116(1):152–162. https://doi.org/10.1007/s11547-010-0587-z

    Article  PubMed  Google Scholar 

  79. Linscott LL, Kessler MM, Kitchin DR et al (2013) CT for pediatric, acute, minor head trauma: clinician conformity to published guidelines. AJNR Am J Neuroradiol 34(6):1252–1256. https://doi.org/10.3174/ajnr.A3366

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  80. Vanderby S, Badea A, Peña Sánchez JN, Kalra N, Babyn P (2018) A day in the life of MRI: the variety and appropriateness of exams being performed in Canada. Can Assoc Radiol J 69(2):151–161. https://doi.org/10.1016/j.carj.2017.05.002

    Article  PubMed  Google Scholar 

  81. Gorleku PN, Dzefi-Tettey K, Edzie EKM et al (2021) The degree and appropriateness of computed tomography utilization for diagnosis of headaches in Ghana. Heliyon. https://doi.org/10.1016/j.heliyon.2021.e06722

    Article  PubMed  PubMed Central  Google Scholar 

  82. Crowell MS, Dedekam EA, Johnson MR, Dembowski SC, Westrick RB, Goss DL (2016) Diagnostic imaging in a direct-access sports physical therapy clinic: a 2-year retrospective practice analysis. Int J Sports Phys Ther 11(5):708–717

    PubMed  PubMed Central  Google Scholar 

  83. Tahvonen P, Oikarinen H, Tervonen O (2020) The effect of interventions on appropriate use of lumbar spine radiograph and CT examinations in young adults and children: a three-year follow-up. Acta Radiol 61(8):1042–1049. https://doi.org/10.1177/0284185119893091

    Article  PubMed  Google Scholar 

  84. Richards PJ, Tins B, Cherian R et al (2002) The emergency department: an appropriate referral rate for radiography. Clin Radiol 57(8):753–758. https://doi.org/10.1053/crad.2002.0970

    Article  CAS  PubMed  Google Scholar 

  85. Sheikh K, Belfi LM, Sharma R, Baad M, Sanelli PC (2012) Evaluation of acute cervical spine imaging based on ACR appropriateness criteria®. Emerg Radiol 19(1):11–17. https://doi.org/10.1007/s10140-011-0994-z

    Article  PubMed  Google Scholar 

  86. Miller JA, Raichlin E, Williamson EE et al (2010) Evaluation of coronary CTA appropriateness criteria® in an academic medical center. J Am Coll Radiol 7(2):125–131. https://doi.org/10.1016/j.jacr.2009.08.013

    Article  PubMed  Google Scholar 

  87. Ryan JW, Hollywood A, Stirling A, Glynn M, MacMahon PJ, Bolster F (2019) Evidenced-based radiology? A single-institution review of imaging referral appropriateness including monetary and dose estimates for inappropriate scans. Ir J Med Sci 188(4):1385–1389. https://doi.org/10.1007/s11845-019-02005-8

    Article  PubMed  Google Scholar 

  88. Pistolese CA, Ciarrapico AM, Della Gatta F, Simonetti G (2013) Inappropriateness of breast imaging: cost analysis. Radiol Med 118(6):984–994. https://doi.org/10.1007/s11547-013-0948-5

    Article  PubMed  Google Scholar 

  89. Squillaci E, Bolacchi F, Scaggiante J et al (2017) Inappropriateness of diagnostic imaging examinations in the inpatient setting: a case study research. Radiol Med (Torino) 122(3):221–227. https://doi.org/10.1007/s11547-016-0708-4

    Article  PubMed  Google Scholar 

  90. Mustard CA, McClarty BM, MacEwan DW (1994) The influence of referral protocols on the utilization of magnetic resonance imaging: evidence from Manitoba. Can Assoc Radiol J 45(2):117–123

    CAS  PubMed  Google Scholar 

  91. Rao S, Rao S, Harvey HB, Avery L, Saini S, Prabhakar AM (2015) Low back pain in the emergency department - Are the ACR appropriateness criteria being followed? J Am Coll Radiol 12(4):364–369. https://doi.org/10.1016/j.jacr.2014.10.020

    Article  PubMed  Google Scholar 

  92. Tambe J, Mbuagbaw L, Nguefack-Tsague G, Foyet J, Ongolo-Zogo P (2020) Multidetector computed tomography utilization in an urban sub-Saharan Africa setting: user characteristics, indications and appropriateness. Pan Afr Med J 37:1–12. https://doi.org/10.11604/pamj.2020.37.42.21176

    Article  Google Scholar 

  93. Hadley JL, Agola J, Wong P (2006) Potential impact of the American College of Radiology appropriateness criteria on CT for trauma. AJR Am J Roentgenol 186(4):937–942. https://doi.org/10.2214/AJR.05.0041

    Article  PubMed  Google Scholar 

  94. Linder R, Horenkamp-Sonntag D, Engel S, Schneider U, Verheyen F (2016) Quality assurance using routine data: overdiagnosis by radiological imaging for back pain. Dtsch Med Wochenschr 141(10):e96–e103. https://doi.org/10.1055/s-0042-101467

    Article  CAS  PubMed  Google Scholar 

  95. Krug B, Boettge M, Reineke T et al (2003) Quality control of outpatient imaging examinations in North Rhine-Westphalia, part II. Rofo 175(3):346–360. https://doi.org/10.1055/s-2003-37823

    Article  CAS  PubMed  Google Scholar 

  96. Klein RP, Velan G, Moscova M, Young N, Shetty A (2021) Quality use of diagnostic imaging in trauma, and the impact on emergency medical practice-a retrospective clinical audit. Emerg Radiol. https://doi.org/10.1007/s10140-021-01921-6

    Article  PubMed  Google Scholar 

  97. Gottlieb RH, Hollenberg GM, Fultz PJ, Rubens DJ (1997) Radiologic consultation: effect on inpatient diagnostic imaging evaluation in a teaching hospital. Acad Radiol 4(3):217–221. https://doi.org/10.1016/S1076-6332(05)80294-6

    Article  CAS  PubMed  Google Scholar 

  98. Griffith B, Kelly M, Vallee P et al (2013) Screening cervical spine CT in the emergency department, phase 2: a prospective assessment of use. AJNR Am J Neuroradiol 34(4):899–903. https://doi.org/10.3174/ajnr.A3306

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  99. Griffith B, Vallee P, Krupp S et al (2014) Screening cervical spine CT in the emergency department, phase 3: Increasing effectiveness of imaging. J Am Coll Radiol 11(2):139–144. https://doi.org/10.1016/j.jacr.2013.05.026

    Article  PubMed  Google Scholar 

  100. Oikarinen H, Karttunen A, Pääkkö E, Tervonen O (2013) Survey of inappropriate use of magnetic resonance imaging. Insights Imaging 4(5):729–733. https://doi.org/10.1007/s13244-013-0276-2

    Article  PubMed  PubMed Central  Google Scholar 

  101. Bianco A, Zucco R, Lotito F, Pavia M (2018) To what extent do hospitalised patients receive appropriate CT and MRI scans? Results of a cross-sectional study in Southern Italy. BMJ Open. https://doi.org/10.1136/bmjopen-2017-018125

    Article  PubMed  PubMed Central  Google Scholar 

  102. Bami K, Premaratne M, Lamba J et al (2017) Appropriate use criteria for cardiac computed tomography: impact on diagnostic utility. J Comput Assisted Tomogr 41(5):746–749. https://doi.org/10.1097/RCT.0000000000000594

    Article  Google Scholar 

  103. Remedios D, Drinkwater K, Warwick R (2014) The clinical radiology audit committee TRCoRL. National audit of appropriate imaging. Clin Radiol 69(10):1039–1044

    Article  CAS  PubMed  Google Scholar 

  104. Gardiner FW, Zhai S (2016) Are all after-hours diagnostic imaging appropriate? An Australian Emergency Department pilot study. Ann Med Surg 12:75–78

    Article  Google Scholar 

  105. Emery DJ, Shojania KG, Forster AJ, Mojaverian N, Feasby TE (2013) Overuse of magnetic resonance imaging. JAMA Intern Med 173(9):823–825

    Article  PubMed  Google Scholar 

  106. Baker M, Jaeger C, Hafley C, Waymack J (2020) Appropriate CT cervical spine utilisation in the emergency department. BMJ Open Qual 9(4):e000844

  107. Hammer M, Mian M, Elhadad L, Li M, Roifman I (2021) Appropriate utilization of cardiac computed tomography for the assessment of stable coronary artery disease. BMC Cardiovasc Disord 21(1):154

  108. Joanna Briggs Institute (2013) JBI levels of evidence: levels of evidence for effectiveness. https://jbi.global/sites/default/files/2019-05/JBI-Levels-of-evidence_2014_0.pdf. Accessed 04 Nov 2022

  109. Geijer H, Geijer M (2018) Added value of double reading in diagnostic radiology, a systematic review. Insights Imaging 9(3):287–301. https://doi.org/10.1007/s13244-018-0599-0

    Article  PubMed  PubMed Central  Google Scholar 

  110. Pow RE, Mello-Thoms C, Brennan P (2016) Evaluation of the effect of double reporting on test accuracy in screening and diagnostic imaging studies: a review of the evidence. J Med Imaging Radiat Oncol 60(3):306–314. https://doi.org/10.1111/1754-9485.12450

    Article  PubMed  Google Scholar 

  111. Shea BJ, Reeves BC, Wells G et al (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

    Article  PubMed  PubMed Central  Google Scholar 

  112. O’Connor C, Joffe H (2020) Intercoder reliability in qualitative research: debates and practical guidelines. Int J Qual Methods 19:1609406919899220. https://doi.org/10.1177/1609406919899220

    Article  Google Scholar 

  113. Waffenschmidt S, Knelangen M, Sieben W, Bühn S, Pieper D (2019) Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med Res Methodol 19(1):132. https://doi.org/10.1186/s12874-019-0782-0

    Article  PubMed  PubMed Central  Google Scholar 

  114. Kahn KL, Park RE, Vennes J, Brook RH (1992) Assigning appropriateness ratings for diagnostic upper gastrointestinal endoscopy using two different approaches. Med Care 30(11):1016–1028

    Article  CAS  PubMed  Google Scholar 

  115. Bami K, Premaratne M, Lamba J et al (2017) Appropriate use criteria for cardiac computed tomography: impact on diagnostic utility. J Comput Assist Tomogr 41(5):746–749. https://doi.org/10.1097/rct.0000000000000594

    Article  PubMed  Google Scholar 

  116. Blachar A, Tal S, Mandel A et al (2006) Preauthorization of CT and MRI examinations: assessment of a managed care preauthorization program based on the ACR appropriateness criteria® and the royal college of radiology guidelines. J Am Coll Radiol 3(11):851–859. https://doi.org/10.1016/j.jacr.2006.04.005

    Article  PubMed  Google Scholar 

  117. Bizzo BC, Almeida RR, Michalski MH, Alkasab TK (2019) Artificial intelligence and clinical decision support for radiologists and referring providers. J Am Coll Radiol 16(9):1351–1356. https://doi.org/10.1016/j.jacr.2019.06.010

    Article  PubMed  Google Scholar 

  118. Doyle J, Abraham S, Feeney L, Reimer S, Finkelstein A (2019) Clinical decision support for high-cost imaging: a randomized clinical trial. PLoS One 14(3):e0213373. https://doi.org/10.1371/journal.pone.0213373

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and Affiliations

Authors

Contributions

Individual contributions to the work according to CRediT: FW: conceptualization, software (lead), investigation (equal), data curation, formal analysis (lead), methodology, visualization, writing—original draft preparation (lead). MEG: project administration, supervision (lead), writing—review & editing (equal). RTH: validation (supporting), supervision, writing—review & editing (equal). JS: methodology (supporting), validation (lead), supervision, writing—review & editing (lead). SFUB: conceptualization, investigation (equal), formal analysis (supporting), methodology, writing—original draft preparation (supporting), writing—review & editing (equal). All authors read and approved the final manscript.

Corresponding author

Correspondence to Felix Walther.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Supplementary material 1 - Original protocol registered at the Open Science Framework. Supplementary material 2 - Applied search strategies. Supplementary material 3 - Full text exclusions with reasons. Supplementary material 4 - Appropriateness results per study.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Walther, F., Eberlein-Gonska, M., Hoffmann, RT. et al. Measuring appropriateness of diagnostic imaging: a scoping review. Insights Imaging 14, 62 (2023). https://doi.org/10.1186/s13244-023-01409-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13244-023-01409-6

Keywords