Table 2 Study characteristics and results

 Yoon LS, USA [13]

Abdominal and pelvic trauma CT

Original report reviewed by a second non-blinded reader

512

30% discordant readings, patient care was changed in 2.3%

Most discordant readings do not result in change in patient care

 Agostini C, France [14]

CT in polytrauma patients

Official interpretation reviewed by two radiologists

105

280 lesions out of 765 (37%) were not appreciated during first reading, of these 31 major

Double reading is recommended in polytrauma patients

 Sung JC, USA [15]

Trauma CT from outside hospital

Re-interpretation by local radiologist

206

12% discrepancies, judged as perceptual in 26% and interpretive in 70%

Double reading is beneficial

 Eurin M, France [16]

Whole-body trauma CT

Scans were re-interpreted for missed injuries by second reader, blinded to initial data

177

157 missed injuries in 85 patients (48%), predominantly minor and musculoskeletal

Double reading is recommended

The second reader missed injuries in 14 patients

 Bechtold RE, USA [17]

Abdominal CT

Clinical report compared with reference standard from a consensus panel

694

56 errors in 694 patients

7.6% errors in CT abdomen, 2.7% clinically significant

 Fultz PJ, USA [18]

CT of ovarian cancer

Four independent readers tested single, single with checklist, paired consensus, and replicated readings

147

Sensitivity for single reader, checklist, paired and replicated readings were 93 to 94% with specificities 79, 80, 82 and 85%, almost all non-significant

The diagnostic aids did not lead to an improved mean observer performance, however an increase in the mean specificity occurred with replicated readings

 Gollub MJ, USA [12]

CT abdomen and pelvis in cancer patients

Original report and re-interpretation report by a non-blinded reader in another hospital was retrospectively compared

143

Major disagreement in 17%, treatment change in 3%

Reinterpretation of body CT scans can have a substantial effect on the clinical care

 Johnson KT, USA [19]

CT colonography with virtual dissection software

Single reading compared with double reading, no consensus

20

Sensitivity/specificity single reading 78–85/80–100%, sensitivity double reading 75–95%

5 mm polyps and larger. No significant increase in sensitivity with double reading

 Murphy R, UK [20]

CT colonography with minimal preparation

Independent and blinded double reading

186

Single reading found 11 cancers and double reading 12, at the expense of 5 false positives for single and 10 for double reading, giving positive predictive values of 69% and 54%, respectively

There is some benefit of double reporting; however, with major resource implications and at the expense of increased false-positives

 Lauritzen PM, Norway [21]

Abdominal CT

Double reading, peer review

1,071

Clinically important changes in 14%

Primary reader chose which studies should be double-read, thus probably more difficult cases. Important changes were made less frequently when abdominal radiologists were first readers, more frequently when they were second readers, and more frequently to urgent examinations

 Wormanns D, Germany [8]

Low-dose chest CT for pulmonary nodules

Independent double reading

9 patients with 457 nodules

Sensitivity of single reading, 54%; double reading, 67%; single reader with CAD, 79%. False positives, 0.9–3.9% for readers, 7.2% for CAD

Double reading and CAD increased sensitivity, CAD more than double reading, at the cost of more false positives for CAD

 Rubin GD, USA [22]

Pulmonary nodules on CT

Independent reading by three radiologists, reference standard by two thoracic radiologists + CAD

20

Sensitivity single reading 50%, double reading 63%, single reading + CAD 76–85%

Double reading increased sensitivity slightly. Inclusion of CAD increased sensitivity further

 Wormanns D, Germany [23]

Chest CT for pulmonary nodules

Independent double reading of low- and standard-dose CT

9 patients with 457 nodules

Sensitivity of single reading, 64%; double reading, 79%; triple reading, 87% (low-dose CT)

Double reading significantly increased sensitivity

5-mm slices used in the study

 Lauritzen PM, Norway [24]

Chest CT

Double reading, peer review

1,023

Clinically important changes in 9%

Primary reader chose which studies should be double-read, thus probably more difficult cases. More clinically important changes were made to urgent examinations, chest radiologists made more clinically important changes than the other consultants

 Lian K, Canada [25]

CT angiography of the head and neck

Blinded double reading by two neuroradiologists in consensus, compared with original report by a neuroradiologist

503

26 significant discrepancies were found in 20 cases, overall miss rate of 5.2%

Double reading may decrease the error rate

 Markus JB, Canada [26]

Double-contrast barium enema

Double and triple reporting, colonoscopy as reference standard

60

Sensitivity/specificity of single reading, 68/96%; double reading. 82/91%

Double reading increased sensitivity and reduced specificity slightly

 Tribl B, Austria [27]

Small-bowel double contrast barium examination in known Crohn’s disease

Clinical report double read by two gastrointestinal radiologists; ileoscopy as reference standard

55

Sensitivity/specificity of single reading, 66/82%; double reading. 68/91%

Negligible improvement by double reading

 Canon CL, USA [28]

Barium enemas, double- and single-contrast

Two independent readers, final diagnosis by consensus. Endoscopy as reference standard

994

Sensitivity/specificity of single reading, 76/91%; simultaneous dual reading, 76/86%

Dual reading led to an increased number of false positives which reduced specificity. No benefit in sensitivity

 Marshall JK, Canada [29]

Small-bowel meal with pneumocolon for diagnosis of ileal Crohn’s disease

Double reading of clinical report by two gastrointestinal radiologists with endoscopy as reference standard

120

Sensitivity/specificity of single reading, 65/90%; double reading, 81/94%

Possibly increased sensitivity with double reading, however unclear information on how study was performed

 Hessel SJ, USA [7]

Chest radiography

Independent reading by eight radiologists, combined by various strategies

100

Pseudo-arbitration was the most effective method overall, reducing errors by 37%, increasing correct interpretations 18%, and adding 19% to the cost of an error-free interpretation

 Quekel LGBA, Netherlands [6]

Chest radiography

Independent and blinded double reading as well as dual reading in consensus

100

Sensitivity/specificity of single reading, 33/92%; independent double reading, 46/87%; simultaneous dual reading, 37/92%

Double or dual reading increased sensitivity and decreased specificity, altogether little impact on detection of lung cancer in chest radiography

 Robinson PJA, UK [30]

Skeletal, chest and abdominal radiography in emergency patients

Independent reading by three radiologists

402

Major disagreements in 5–9% of cases

The magnitude of interobserver variation in plain film reporting is considerable

 Soffa DJ, USA [31]

General radiography

Independent double reading by two radiologists

3,763

Significant disagreement in 3%

Part of a quality assurance program

 Wakeley CJ, UK [32]

MR imaging

Double reading by two radiologists. Arbitration in case of disagreement

100

9 false-positive, 14 false-negative reports in 100 cases

The study promotes the benefits of double reading MRI studies

 Siegle RL, USA [33]

General radiology in six departments, including CT, nuclear medicine and ultrasound

Double reading by a team of QC radiologists

11,094

Mean rate of disagreement 4.4% in over 11,000 images

Rates of disagreement lower than previously reported

 Warren RM, UK [34]

MR breast imaging

Blinded and independent double reading by two observers, 44 in total!

1,541

Sensitivity/specificity of single reading, 80/88%; double reading, 91/81%

Double reading increased sensitivity at the cost of decreased specificity

 Babiarz LS, USA [35]

Neuroradiology cases

Original report by neuroradiologist, double reading by another neuroradiologist

1,000

2% rate of clinically significant discrepancies

Low rate of disagreements, but all worked in the same institution

 Agrawal A, India [36]

Teleradiology emergency radiology

Parallel dual reporting

3,779

3.8% error rate, CT abdomen and MRI head/spine most common error sources

Focused double read of pre-identified complex, unfamiliar or error-prone case types may be considered for optimum utilisation of resources

 Harvey HB, USA [37]

CT, MRI and ultrasound

Peer review using consensus-oriented group review

11,222

Discordance in 2.7%, missed findings most common

Highest discordance rates in musculoskeletal and abdominal divisions

 Kalbhen CL, USA [38]

Abdominal CT for pancreatic carcinoma

Original report reviewed by sub-specialty radiologists

53

32% discrepancies in 53 patients, all under-staging

Reinterpretation of outside abdominal CT was valuable for determining pancreatic carcinoma resectability

 Tilleman EH, Netherlands [39]

CT or ultrasound in patients with pancreatic or hepatobiliary cancer

Reinterpretation by sub-specialised abdominal radiologist

78

48% of ultrasound and 30% of CT studies were judged as not sufficient for reinterpretation

Change in treatment strategy in 9%. Many initial reports were incomplete

Major discordance in 8% for ultrasound, 12% for CT

 Bell ME, USA [40]

After-hours body CT

Abdominal imaging radiologists reviewed reports by non-sub-specialists

1,303

4.4% major discrepancies in 742 cases double read by primary members of the abdominal imaging division, 2.0% major discrepancies in 561 cases double read by secondary members

The degree of sub-specialisation affects the rate of clinically relevant and incidental discrepancies

 Lindgren EA, USA [5]

CT, MR and ultrasound from outside institutions submitted for secondary interpretation

Second opinion by sub-specialised GI radiologist

398

5% high clinical impact and 7.5% medium clinical impact discrepancies

The second reader had 2% medium clinical impact discrepancies. There was a trend towards overcalls in normal cases and misses in complicated cases with pathology

 Wibmer A, USA [41]

Diagnosis of extracapsular extension of prostate cancer on MRI

Second-opinion reading by sub-specialised genitourinary oncological radiologists

71

Disagreement between the initial report and the second-opinion report in 30% of cases, second-opinion correct in most cases

Reinterpretation by sub-specialist improved detection of extracapsular extension

 Rahman WT, USA [42]

Abdominal MRI in patients with liver cirrhosis

Re-interpretation by sub-specialised hepatobiliary radiologist

125

10% of subjects had a discrepant diagnosis of hepatocellular cancer, and 10% of subjects had discrepant Milan status for transplant

Reinterpretations were more likely to describe imaging findings of cirrhosis and portal hypertension and more likely to make a definitive diagnosis of HCC

50% change in management

 Cascade PN, USA [43]

Chest radiography

Performance of chest faculty and non-chest radiologists was evaluated

485,661

No difference in total rate of incorrect diagnoses, but non-chest faculty had a statistically significant higher rate of seemingly obvious misdiagnoses

There are several potential biases in the study which complicate the conclusions

 Nordholm-Carstensen A, Denmark [44]

Chest CT in colorectal cancer patients, classification of indeterminate nodules

Second opinion by sub-specialised thoracic radiologist

841

Sensitivity/specificity primary reading 74/99%, sub-specialist 92/100%

Higher sensitivity for the thoracic radiologist with fewer indeterminate nodules

 Jordan MJ, USA [45]

Emergency head CT

Original report reviewed by sub-specialty neuroradiologists

1,081

4 (0.4%) clinically significant and 10 insignificant errors

Double reading of head CT by sub-specialist appears to be inefficient

 Briggs GM, UK [46]

Neuro CT and MR

Second opinion by sub-specialised neuro-radiologist

506

13% major discrepancy rate

The benefit of a formal specialist second opinion service is clearly demonstrated

 Zan E, USA [47]

Neuro CT and MR

Reinterpretation by sub-specialised neuroradiologist

4,534

7.7% of clinically important differences

Double reading is recommended

When reference standards were available, the second-opinion consultation was more accurate than the outside interpretation in 84% of studies

 Jordan YJ, USA [48]

Head CT, stroke detection

Original report reviewed by sub-specialty neuroradiologists

560

0.7% rate of clinically significant discrepancies

Low rate of discrepancies and double reading by sub-specialist was reported as inefficient. However the study was limited to ischaemic non-haemorrhagic disease

 Eakins C, USA [49]

Paediatric radiology

Cases referred to a children’s hospital were reviewed by a paediatric sub-specialist

773

22% major disagreements

Interpretations by sub-specialty radiologists provide important clinical information

When final diagnosis was available, the second interpretation was more accurate in 90% of cases

 Bisset GS, USA [50]

Paediatric extremity radiography

Official interpretation reviewed by one paediatric radiologist, blinded to official report. Arbitration by a second radiologist when reports differed

3,865

Diagnostic errors in the form of a miss or overcall occurred in 2.7% of the radiographs

Diagnostic errors quite rare in paediatric extremity radiography. Clinical significance of the discrepancies was not evaluated

 Onwubiko C, USA [51]

CT abdomen in paediatric trauma patients

Re-review of images by paediatric radiologist

98

12.2% new injuries identified, 3% had solid organ injuries upgraded, and 4% downgraded to no injury

Clear benefit to having referring hospital trauma CT scans reinterpreted by paediatric radiologists

 Loevner LA, USA [52]

CT and MR in head and neck cancer patients

Second opinion by sub-specialised neuroradiologist

136

Change in interpretation in 41%, TNM change in 34%, mostly up-staging

Sub-specialist increases diagnostic accuracy

 Kabadi SJ, USA [53]

CT, MR and ultrasound from outside institutions submitted for formal over-read

Retrospective review

362

12.4% had clinically significant discrepancies

64% perceptual errors

Strategies for reducing errors are suggested