Skip to main content

Table 3 Evaluation performance of generative methods presented in the examined studies for brain tumors

From: Enhancing cancer differentiation with synthetic MRI examinations via generative models: a systematic review

Author

Quantitative metrics

Qualitative metrics

Downstream task

 

Direct for SD

Indirect prior to SDA

Indirect after SDA

Experts/statistics(%)

Model

Beers [38]

N/A

N/A

N/A

Plotted samples

N/A

Han [39]

N/A

N/A

N/A

Physician/53.0

N/A

Han [41]

N/A

Acc: 0.900

Sen: 0.852

Spe: 0.970

Acc: 0.910

Sen: 0.866

Spe: 0.976

(& classical DA)

Physician/78.5

ResNet-50

Han [42]

N/A

Acc: 0.931

Sen: 0.909

Spe: 0.958

Acc: 0.948, Sen: 0.936, Spe: 0.984

(GAN-based DA)

Acc: 0.967, Sen: 0.974, Spe: 0.988

(GAN-based & classic DA)

Physician/76.0

t-SNE

ResNet-50

Han [44]

N/A

Sen: 0.67

Sen: 0.77

Three Physicians/

Ph1: 91.0, Ph2: 96.0, Ph3: 100

t-SNE

YOLOv3

Shin [46]

N/A

Dice: 0.64 ± 0.14

Dice: 0.80 ± 0.07

Plotted samples

pix2pix

Chang [47]

N/A

Dice: 0.748

Sen: 0.798

Spe: 0.995

HD(95): 12.85

Dice: 0.704

Sen: 0.729

Spe: 0.995

HD(95): 14.94

Plotted samples

U-Net

Chang [48]

N/A

Dice: 0.808

Sen: 0.785

Spe: 0.996

HD(95): 11.95

Dice: 0.773

Sen: 0.742

Spe: 0.996

HD(95): 16.44

Plotted samples

U-Net

Deepak [49]

N/A

Balanced Acc: 0.903

Balanced Acc: 0.931

Plotted samples

CNN

Qasim [50]

N/A

N/A

Dice: 0.779

Plotted samples

U-Net

Kwon [52]

MMD: 0.072

MS-SSIM: 0.843

N/A

N/A

PCA

N/A

Chen [54]

IS: 2.32 ± 0.04

FID: 139 ± 4.5

KID 0.144 ± 0.006

Acc: 0.901

Acc: 0.888

Plotted samples

CNN

Pesteie [55]

N/A

Dice: 0.80 ± 0.33

Hausdorff: 2.78 ± 3.24

Dice: 0.88 ± 0.26

Hausdorff: 2.16 ± 2.6

Plotted samples

U-Net

Hamghalam [56]

SSIM: 0.7245

PSNR: 22.23

N/A

Dice: up-to 0.89

Sen: up-to 0.96

PPV: up to 0.83

Plotted samples

Pixelwise

Classifier

Qi [57]

N/A

Acc: up-to 0.933

AUC: up-to 0.961

Acc: up-to 0.950

AUC: up-to 0.969

Plotted samples

ResNet-18

Guo [58]

N/A

Dice: up-to 0.673

HD(95): as low as 7.078

Sen: up-to 0.678

Spe: 0.999

Dice: up to 0.821

HD(95): as low as 1.568

Sen: up-to 0.807

Spe: 0.999

Neuroradiologist/72.1

U-Net

Guo [60]

Pixel Acc: up-to 0.774

SSIM: 0.812

PNSR: 21.8

Dice: up-to 0.672

Dice: up-to 0.840

Plotted samples

U-Net

Ge [66]

PSNR: up-to 26.14

DAEF: as low as 132.40

Acc: 0.852 ± 0.322

Sen: 0.690 ± 0.137

Spe: 0.939 ± 0.389

Acc: 0.888 ± 0.637

Sen: 0.818 ± 0.111

Spe: 0.921 ± 0.477

Radiologist/-

Multi-stream

2D CNN

Ge [67]

N/A

Acc: 0.853 ± 0.443, Sen: 0.735 ± 0.927

Spe: 0.909 ± 0.525

Acc: 0.895 ± 0.142, Sen: 0.782 ± 0.435

Spe: 0.936 ± 0.275

Acc: 0.865 ± 0.424, Sen: 0.737 ± 0.815

Spe: 0.927 ± 0.345

Acc: 0.907 ± 0.142, Sen: 0.843 ± 0.659

Spe: 0.930 ± 0.142

Plotted samples

Multi-stream

2D CNN

Carver [68]

MSE: as low as 18.4

MAE: as low as 22.8

SSIM: up-to 0.794

PSNR: 43.1

N/A for Dice

Sen: up-to 0.89, Spec: up-to 0.99

Dice: increase of 0.48

Sen: up-to 0.90, Spec: up-to 0.99

Physician/26.3

U-Net

Mok [69]

N/A

Dice: 0.79

Dice: 0.84 (GAN-Based DA)

Dice: 0.81 (classical DA)

Plotted samples

U-Net

Dikici [72]

FD > 0.4

AFP: 9.12

AFP: 9.53

t-SNE

BM-detection

framework

Kamli [75]

N/A

Recall: 0.643

Precision: 0.625

Dice: 0.641

Recall: 0.699

Precision: 0.717

Dice: 0.723

Plotted samples

TGP

Li [77]

N/A

N/A

Acc: 0.920

AUC: 0.947

t-SNE

SVM

  1. SD, Synthetic Data; SDA, Synthetic Data Augmentation; Acc, Accuracy; Sen, Sensitivity; Spe, Specificity; t-SNE, t-distributed Stochastic Neighbor Embedding; HD(95), Hausdorff Distance; AUC, Area Under the Curve; SSIM, Structural Similarity Index Measure; PSNR, Peak Signal-to-Noise Ratio; DAEF, Distance to the real images based on Autoencoder Features; PPV, Predicted Positive Value; MSE, Mean Square Error; MAE, Mean Absolute Error; FD, Frechet Distance; AFP, Average False Positive; MMD, Maximum Mean Discrepancy; MS-SSIM, Multi-Scale Structural Similarity Metric; PCA, Principal Component Analysis; IS, Inception Score; FID, Frechet Inception Distance; KID, Kernel Inception Distance; N/A, Not applicable; Symbol ’-’ represents that the corresponding information was not provided in the publication