Skip to main content

Table 4 Performance metrics for the experienced and inexperienced radiologists with and without model assistance in the comparison set

From: Deep learning based on ultrasound images assists breast lesion diagnosis in China: a multicenter diagnostic study

  AUC (95%CI) p value Sensitivity (95%CI) p value Specificity (95%CI) p value PPV (95%CI) p value NPV (95%CI) p value ACC (95%CI) p value
Radiologists without DL assistance             
All 0.843 (0.819–0.865)  < 0.0001* 96.82 (94.72–98.25)  < 0.0001* 42.48 (38.36–46.67)  < 0.0001* 56.72 (54.93–58.50)  < 0.0001* 94.49 (91.03–96.66) 0.1065 66.27 (63.25–69.19)  < 0.0001*
Ex 0.919 (0.888–0.944) 0.6778 96.59 (92.73–98.74) 0.0118* 63.27 (56.63–69.57)  < 0.0001* 67.19 (63.27–70.90) 0.0029* 95.97 (91.52–98.14) 0.0774 77.86 (73.48–81.83) 0.0009*
Inex 0.798 (0.764–0.830)  < 0.0001* 96.97 (94.12–98.68) 0.0005* 28.61 (23.86–33.75)  < 0.0001* 51.41 (49.64–53.17)  < 0.0001* 92.38 (85.72–96.08) 0.7031 58.54 (54.49–62.51)  < 0.0001*
Radiologists with DL assistance method one             
All 0.861 (0.838–0.881)  < 0.0001# 97.27 (95.28–98.58) 0.8036 64.25 (60.14–68.21)  < 0.0001# 67.94 (65.46–70.32)  < 0.0001# 96.80 (94.52–98.15) 0.1533 78.71 (76.04–81.20)  < 0.0001#
Ex 0.932 (0.903–0.954) 0.1044 96.59 (92.73–98.74) 1.0000 80.09 (74.28–85.09)  < 0.0001# 79.07 (74.39–83.09) 0.0041# 96.79 (93.20–98.52) 0.6885 87.31 (83.66–90.41)  < 0.0001#
Inex 0.819 (0.786–0.849)  < 0.0001# 97.73 (95.12–99.16) 0.7266 53.69 (48.22–59.09)  < 0.0001# 62.17 (59.40–64.86) 0.0011# 96.81 (93.18–98.54) 0.0890 72.97 (69.23–76.48)  < 0.0001#
Radiologists with DL assistance method two             
All 0.908 (0.888–0.925)  < 0.0001# 97.73 (95.86–98.91) 0.4545 66.90 (62.85–70.77)  < 0.0001# 69.69 (67.14–72.13)  < 0.0001# 97.42 (95.33–98.59) 0.0555 80.40 (77.81–82.81)  < 0.0001#
Ex 0.933 (0.904–0.955) 0.1202 97.16 (93.49–99.07) 1.0000 75.66 (69.53–81.11)  < 0.0001# 75.66 (71.16–79.67) 0.0412# 97.16 (93.49–98.79) 0.5564 85.07 (81.21–88.41) 0.0001#
Inex 0.902 (0.875–0.924)  < 0.0001# 98.11 (95.64–99.38) 0.5488 61.06 (55.65–66.28)  < 0.0001# 66.24 (63.17–69.18)  < 0.0001# 97.64 (94.54–99.00) 0.0265# 77.28 (73.72–80.57)  < 0.0001#
  1. AUC area under the receiver operating characteristic curve, PPV positive predictive value, NPV negative predictive value, ACC accuracy, CI confidence interval, DL deep learning, All all the five radiologists, Ex experienced radiologists, Inex inexperienced radiologists
  2. *p values are that of radiologists without DL assistance versus DL and show significant difference
  3. #p values are that of radiologists with DL assistance vs. radiologists without DL assistance and show significant difference