Skip to main content

Table 4 Performance metrics for the experienced and inexperienced radiologists with and without model assistance in the comparison set

From: Deep learning based on ultrasound images assists breast lesion diagnosis in China: a multicenter diagnostic study

 

AUC (95%CI)

p value

Sensitivity (95%CI)

p value

Specificity (95%CI)

p value

PPV (95%CI)

p value

NPV (95%CI)

p value

ACC (95%CI)

p value

Radiologists without DL assistance

            

All

0.843 (0.819–0.865)

 < 0.0001*

96.82 (94.72–98.25)

 < 0.0001*

42.48 (38.36–46.67)

 < 0.0001*

56.72 (54.93–58.50)

 < 0.0001*

94.49 (91.03–96.66)

0.1065

66.27 (63.25–69.19)

 < 0.0001*

Ex

0.919 (0.888–0.944)

0.6778

96.59 (92.73–98.74)

0.0118*

63.27 (56.63–69.57)

 < 0.0001*

67.19 (63.27–70.90)

0.0029*

95.97 (91.52–98.14)

0.0774

77.86 (73.48–81.83)

0.0009*

Inex

0.798 (0.764–0.830)

 < 0.0001*

96.97 (94.12–98.68)

0.0005*

28.61 (23.86–33.75)

 < 0.0001*

51.41 (49.64–53.17)

 < 0.0001*

92.38 (85.72–96.08)

0.7031

58.54 (54.49–62.51)

 < 0.0001*

Radiologists with DL assistance method one

            

All

0.861 (0.838–0.881)

 < 0.0001#

97.27 (95.28–98.58)

0.8036

64.25 (60.14–68.21)

 < 0.0001#

67.94 (65.46–70.32)

 < 0.0001#

96.80 (94.52–98.15)

0.1533

78.71 (76.04–81.20)

 < 0.0001#

Ex

0.932 (0.903–0.954)

0.1044

96.59 (92.73–98.74)

1.0000

80.09 (74.28–85.09)

 < 0.0001#

79.07 (74.39–83.09)

0.0041#

96.79 (93.20–98.52)

0.6885

87.31 (83.66–90.41)

 < 0.0001#

Inex

0.819 (0.786–0.849)

 < 0.0001#

97.73 (95.12–99.16)

0.7266

53.69 (48.22–59.09)

 < 0.0001#

62.17 (59.40–64.86)

0.0011#

96.81 (93.18–98.54)

0.0890

72.97 (69.23–76.48)

 < 0.0001#

Radiologists with DL assistance method two

            

All

0.908 (0.888–0.925)

 < 0.0001#

97.73 (95.86–98.91)

0.4545

66.90 (62.85–70.77)

 < 0.0001#

69.69 (67.14–72.13)

 < 0.0001#

97.42 (95.33–98.59)

0.0555

80.40 (77.81–82.81)

 < 0.0001#

Ex

0.933 (0.904–0.955)

0.1202

97.16 (93.49–99.07)

1.0000

75.66 (69.53–81.11)

 < 0.0001#

75.66 (71.16–79.67)

0.0412#

97.16 (93.49–98.79)

0.5564

85.07 (81.21–88.41)

0.0001#

Inex

0.902 (0.875–0.924)

 < 0.0001#

98.11 (95.64–99.38)

0.5488

61.06 (55.65–66.28)

 < 0.0001#

66.24 (63.17–69.18)

 < 0.0001#

97.64 (94.54–99.00)

0.0265#

77.28 (73.72–80.57)

 < 0.0001#

  1. AUC area under the receiver operating characteristic curve, PPV positive predictive value, NPV negative predictive value, ACC accuracy, CI confidence interval, DL deep learning, All all the five radiologists, Ex experienced radiologists, Inex inexperienced radiologists
  2. *p values are that of radiologists without DL assistance versus DL and show significant difference
  3. #p values are that of radiologists with DL assistance vs. radiologists without DL assistance and show significant difference