Analysis of computer-aided diagnostics in the preoperative diagnosis of ovarian cancer: a systematic review

Koch, Anna H.; Jeelof, Lara S.; Muntinga, Caroline L. P.; Gootzen, T. A.; van de Kruis, Nienke M. A.; Nederend, Joost; Boers, Tim; van der Sommen, Fons; Piek, Jurgen M. J.

doi:10.1186/s13244-022-01345-x

Insights into Imaging

Table 1 Results depicted per image modality: ultrasound, CT and MRI

From: Analysis of computer-aided diagnostics in the preoperative diagnosis of ovarian cancer: a systematic review

Included studies	Study setting	Patients (n)	Samples (n)	CAD-model	Features (n)	Performance ACC	Performance AUC	Performance Sensitivity	Performance Specificity	Performance Other	CAD model evaluation method	Compared to other models or reviewer(s)*
a: CAD ultrasound (22)
Gao et al. [33]	Retrospective Case–control	107,624	575,930 images	DCNN	121 layers	(1) 86.9%	(1) 0.870)	(1) 40.3%	(1) 91.6%	Brier-score	1 internal validation set	Radiologist alone (3) Radiologist with DCNN (4)
			103,370 benign			(2) 85.3%	(2) 0.831	(2) 57.8%	(2) 98.5%	F1-score	2 external validation set (1 + 2)
			4254 malignant			(3) 81.1%	(3) N/A	(3) 55.5%	(3) 87.5%	PPV
						(4) 87.6%	(4) N/A	(4) 82.7%	(4) 88.7%	NPV
Chiappa et al. [34]	Retrospective Case–control	241	241 images	SVM	853	80.00%	0.83	78.00%	83.00%	N/A	Training-Validation Testing Nested-tenfold validation	N/A
			115 benign		269 solid	87.00%	0.88	75.00%	90.00%
			126 malignant		278 cystic	81.00%	0.89	81.00%	81.00%
					306 motley
Chiappa et al. [35]	Retrospective & Prospective Case–control	274	274 images	DSS	857	(1) 87.9%	N/A	(1) 99.2%	(1) 75.9%	PPV	External validation in prospective cohort (n = 35) tenfold cross validation	2 gynecologists with DCNN (1 + 2) on internal & external dataset
		239	239		269 solid	(2) 88.7%		(2) 98.4%	(2) 78.5%	NPV
		35	123 benign		278 cystic	(1) 91.4%		(1) 100.0%	(1) 80.0%
			116 malignant		306 motley	(2) 91.4%		(2) 100.0%	(2) 80.0%
			35		4 clinical
			15 benign
			20 malignant
Christiansen et al. [36]	Retrospective Case–control	758	3077	VGG16	1024 layers	91.30%	0.95	96.00%	86.70%	N/A	Training 67%	SA (2)
		634 surgery	1927 grayscale	ResNet	512 layers	(2) 92.0%	N/A	(2) 96.0%	(2) 88.0%		Validation 13%	RMI (3)
		124 follow-up	1150 power doppler	MobileNet		(3) 93.6%		(3) 94.5%	(3) 92.6%		Testing 20%	SR (4)
				(Ovry-Dx1)		(4) 96.0%		(4) 66.7%	(4) 81.3%			SRR (5)
			449 benign
			309 malignant
Qi et al. [37]	Retrospective Case–control	265	279 images	Nomogram with LASSO and RADscore	17	(1) 88.0%	(1) 0.914	(1) 81.3%	(1) 92.2%	IDI	Training 70% Validation 30% tenfold cross validation	Senior (3) & Junior (4) sonographists
			106 benign	task 1 + 2	22	(2) 86.3%	(2) 0.890	(2) 84.2%	(2) 97.5%			task 1 benign – malignant (1)
			65 borderline tumors		4 clinical	(3) 79.5%	(3) 0.789	(3) 69.7%	(3) 86.0%			task 2 benign-borderline-malignant (2)
			108 malignant tumors			(3) 64.7%	(3) 0.612	(3) 53.6%	(3) 68.6%
						(4) 69.9%	(4) 0.669	(4) 56.8%	(4) 80.4%
						(4) 56.9%	(4) 0.521	(4) 56.3%	(4) 62.2%
Stefan et al. [63]	Retrospective Case–control	120	123 images	KNN	3	85.37%	N/A	80.00%	87.50%	PPV	Run KNN twice**	N/A
			85 benign
			35 malignant
Wang et al. [38]	Retrospective Case–control	265	279 images 108 benign	VGG	N/A	(1) 91.4%	(1) 0.963	(1) 91.4%	(1) 91.4%	F1-score	Transfer learning	Sonographist (3) task C (benign-borderline-malignant)
			65 borderline 106 malignant	GoogleNet		(2) 75.3%	(2) N/A	(2) 80.0% / 45.5% / 88.9%	(2) 89.7% / 95.8% / 75.4%		threefold-cross validation	task A benign – malignant (1)
				ResNet		(3) 66.7%	(3) N/A	(3) 75.0% / 47.4% / 68.4%	(3) 81.8% / 85.2% / 82.5%
				MobileNet
				task A + C (1) + (2)
Martinez-Mas et al. [39]	Retrospective Case–control	187	384 images 112 benign	SVM	N/A	87.70%	0.874	92.00%	80.00%	N/A	LOO-CV	N/A
			75 malignant	KNN							N = 30
				LD
				ELM
Zhang et al. [40]	Retrospective Case–control	N/A	428 images 357 malignant 71 benign 1400 images 277 malignant 299 benign	Cost-sensitive RF	N/A	99.20%	0.997	99.70%	95.60%	N/A	Transfer learning	N/A
				VGGNet							Training 71.5%
				GoogleNet							Validation 14.3%
				FCNN							Testing 14.3%
				AlexNet							tenfold-cross validation
Acharya et al. [41]	N/A	469	469	KNN	39	80.60%	0.806	81.40%	76.30%	N/A	tenfold cross validation	N/A
	Cohort		238 suspicious 281 non-suspicious	RF
				FF
				FRNN
Aramendia-Vidaurreta et al. [46]	N/A	145	145 images 106 benign	MLP	40	98.80%	0.997	98.50%	98.90%	PPV	Training 80% Validation 10% Testing 10%	N/A
	Case–control		39 malignant		1 clinical						tenfold cross validation
											40:30:01
Khazendar et al. [47]	Retrospective Cohort	177	187 images	SVM	1	78.00%	N/A	80.00%	77.00%	T-test	Training and testing set	N/A
			112 benign	LBP on enhanced image							50-fold cross validation Performance of the SVM per 15 cycles
			75 malignant
Acharya et al. *** [44]	Retrospective Case–control	20	10 benign	SVM	11	(1) 100%	N/A	(1) 100%	(1) 100%	N/A	Training and testing set	N/A
			10 malignant 2600 images 1300 benign 1300 malignant	KNN		(2) 100%		(2) 100%	(2) 100%		tenfold cross validation
				PNN
Acharya et al. **** [42]	Prospective cohort	23	20	PNN	23	99.81%	N/A	99.92%	99.69%	PPV	Training 90%	N/A
			10 benign								Testing 10%
			10 malignant								tenfold-cross validation
			2600 images
			1300 benign
			1300 malignant
Acharya et al. [45]	Prospective Case–control	10	20	DT	4	N/A	N/A	94.30%	99.70%	PPV	Training and testing set	N/A
			10 benign							TP rate	tenfold cross validation
			10 malignant							FP rate
										TN rate
			2000 images							FN rate
			1000 benign
			1000 malignant
Faschingbauer et al. [48]	Retrospective Case–control	105	105	SVM-ABTA	(1) 16	(1) N/A	N/A	(1) 69%	(1) 86%	Youden-index	Training and testing set	Level III gynaecologists (5)
			70 benign	Malignant (1)	(2) 16	(2) N/A		(2) 72%	(2) 81%		onefold cross validation
			35 malignant	Dermoid cysts (2)	(3) 16	(3) N/A		(3) 82%	(3) 96%
				Functional cysts (3) Overall (4)	(4) 16	(4) 74.3%		(4) N/A	(4) N/A
						(5) 83.75%
Acharya et al. [43]	Retrospective Cohort	20	20	SVM-RBF	14	99.90%	N/A	100%	99.80%	PPV	Training and testing set	N/A
			10 benign							TP rate	tenfold cross validation
			10 malignant							FP rate
										TN rate
			2000 images 1000 benign 1000 malignant							FN rate
Vaes et al. [49]	Prospective Case–control	197	291 adnexal masses	OVHS + RMI1	N/A	N/A	N/A	88%	95%	N/A	Training 70%	N/A
			125 benign	OVHS + RMI2							Testing 30%
			166 malignant	OVHS + RMI3							100 times a random subsampling process
Vaes et al. [50]	Prospective Case–control	197	197 ultrasound images—365 ovarian tumors	LR (1)	(1) 9	N/A	(1) 0.97	(1) 83%	(1) 98%	N/A	Training 60%	RMI (3)
			77—normal	NN (2)	(2) N/A		(2) 0.93	(2) 80%	(2) 86%		Testing 40%	LR2 (4)
			125—benign		(3) 7		(3) 0.80	(3) 69%	(3) 79%		100 bootstrap resampled data sets with AICC selection	NN2 (5)
			166—malignant		(4) 6		(4) 0.85	(4) 79%	(4) 70%
					(5) 7		(5) 0.87	(5) > 99%	(5) 10%
Lucidarme et al. [52]	Prospective Case–control	264	375 ovaries	OVHS	N/A	N/A	N/A	98%	88%	PPV	One group	N/A
			107 normal							NPV
			127 benign							TP rate
			141 malignant							FP rate
										TN rate
			359 sonographist opinion							FN rate
			104 normal ovaries
			119 benign
			136 malignant
Lu et al. [51]	N/A	425	425	SVM (1)	(1) 10	(1) 84.38%	(1) 0.918	(1) 85.19%	(1) 83.96%	PPV	Training 62%	RMI (2)
	Case control		291 benign		(2) 7	(2) 76.88&	(2) 0.873	(2) 81.48%	(2) 74.53%	NPV	Testing 38%	LR1 (3)
			134 malignant		(3) 12	(3) 80.63%	(3) 0.911	(3) 81.48%	(3) 80.19%		1 internal test set	LR2 (4)
					(4) 6	(4) 78.75%	(4) 0.916	(4) 81.48%	(4) 77.36%		1 external validation set
											30-fold cross validation
Zimmer et al. [53]	Retrospective Case–control	163	163 images	Bayes method	4	82.10%		80%	100%	PPV	Training 85%	N/A
			25 transparent cyst							NPV	External validation 15%
			67 turbid cyst
			50 significantly solid
			21 solid
b: CAD CT (3)
Li et al. [54]	Retrospective Case–control	140	140	Radiomics segmentation models	(1) 10	(1) 97.6%	(1) 0.99	(1) 95.7%	(1) 100%	N/A	Training 61%	N/A
			62 benign		4 clinical	(2) 90.2%	(2) 0.97	(2) 100%	(2) 82.6%		Testing 29%
			72 malignant		(2) 11						07:03
					5 clinical
Park et al. [55]	Retrospective Case–control	427	427	RF	8	N/A	0.88	91%	69%	N/A	tenfold cross validation	N/A
			348 benign	LR
			79 malignant
Li et al. [56]	Retrospective Case–control	160	160 images	Nomogram (int. val)	14	(1) 89.7%	(1) 0.897	(1) 94.7%	(1) 85.0%	N/A	Training 59%	N/A
			134	Nomogram (ext. val)							Testing 24%
			62 benign			(2) 88.0%	(2) 0.880	(2) 84.6%	(2) 91.7%		External validation 17%
			72 malignant								tenfold cross validation
			External dataset N/A
c: CAD MRI (6)
Liu et al. [57]	Retrospective Case–control	196	196	Radiomics segmentation	(1) 396	(1) 99,0%	(1) 1.0	(1) 100%	(1) 98.0%	PPV	Random Training 50% Testing 50%	N/A
			91 borderline	models*						NPV
			10 malignant	3D sagit (1)	(2) 396	(2) 78.9%	(2) 0.82	(2) 72.9%	(2) 85.1%
				2D coron (2)
Song et al. [58]	Prospective Case–control	82	104	PK-model	(1) 7	(1) 84.2%	N/A	(1) 66.7%	(1) 100%	N/A	Training 70% Validation 30% 3-class classification task	Radiologists (2)
			33 benign		(2) N/A	(2) 68.4%		66.70%	93.80%		50-fold cross-validation	benign
			18 borderline					70%	77.80%			borderline
			53 malignant					(2) 66.7%	(2) 92.3%			malignant
								66.70%	81.3%%
								70%	77.80%
Jian et al. ** [59]	Retrospective Case–control	501	501	MICNN	512	76.70%	0.884	74.80%	80.80%	F1 score	Training 68%	N/A
			165 borderline	EMP							(centers A-B)
			336 malignant	LMP							External validation set 32%
											(centers C-H)
Jian et al. *** [62]	Retrospective Case–control	501	22,977	MICNN MAC-net	512	82.70%	0.878	N/A	N/A	F1 score	Training 76% Validation 23%	N/A
			501
			165 borderline
			336 malignant
Li et al. [61]	Retrospective Case–control	501	501	MP-ST (1)	(1) 851	(1) N/A	(1) 0.920	(1) N/A	(1) N/A	N/A	Training 50%	Radiologists (3)
			165 borderline	CE-T1W1 (2)	(2) 851	(2) N/A	(2) 0.801	(2) N/A	(2) N/A		Internal validation 18%
			336 malignant		(3) N/A	(3) N/A	(3) 0.797	(3) 80.5%	(3) 78.9%		(centers A-B)
											External validation 32% (centers C-H)
Zhang et al. [60]	Retrospective Case–control	280	72 benign	SVM (b-m) (1)	(1) 84	(1) 90.6%	(1) 0.9670	(1) 90.3%	(1) 91.3%	PPV	Randomly	Radiologists (3)
			100 type I EOC	SVM (I-II) (2)	(2) 56	(2) 83.3%	(2) 0.8228	(2) 76.5%	(2) 86.5%	NPV	LOOCV 70%
			81 type 2 EOC		(3) N/A	(3) 83.5%	(3) N/A	(3) 82.3%	(3) 86.9%	TP rate	Testing 30%
										FP rate
										TN rate
										FN rate

AUC = Area Under the Curve; PPV = positive predictive value; NPV = negative predictive value;SVM = standard vector machine; DCNN = (deep) Convolutional Neural Network); N/A = not applicable; DSS = decision; support system, based on 3 radiomics models VGGNet, ResNet, MobileNet; SA = subjective assessment of an expert (gynaecologist/sonographist); SR = IOTA Simple Rules model; SRL = IOTA simple rules risk model; IDI = integrated discrimination improvement; KNN = k-nearest neighbor; LD = Linear Discriminant; ELM = Extreme Machine Learning (***linear-gaussian in this example); LOO-CV = Leave-One-Out Cross Validation procedure; FCNN = Fully Connected Convolutional Neural Network; RF = Random Forest; FRNN = Fuzzy-Rough Nearest Neighbor; FF = fuzzy forest; MLP Multilayer Perceptron Networks; LBP = Local Binary Pattern; PNN = Probabilistic Neural Network; DT = Decision Tree; ABTA = automatic texture based algorithm; RBF = Radial Basis Function; OVHS = Ovarian HIstoscanning; RMI = Risk of Malignancy; LR = Logistic Regression; NN = Neural Network; AICC = Akaike information corrected criterion; Bold = best performing classifier
N/A = not applicable; LR = logistic regression; RF = random forest; Bold = best performing classifier
MICNN = Multiple instance convolutional neural network; EMP = early multiparametric; LMP = late multiparametric; PK model = pharmacokinetic model; MP-solid = multiparametric solid tumor model; CE-T1W1 = Contrast-enhanced T1W1 model; Bold = best performing
* = radiologist, gynaecologist, sonographist or other(s)
** = Unable to split data set in 70% and 30% training and validation sets, due to limited number malignant tumors, therefore classifier was run twice with different variables
*** = Acharya et al. [44—GyneScan: An improved online paradigm for screening of ovarian cancer via tissue characterization
**** = Acharya et al.[42]—Evolutionary algorithm-based classifier parameter tuning for automatic ovarian cancer tissue characterization and classification
* Different segmentation models were constructed using 3D and 2D MRI in coronal and sagittal plane;
**Jian et al. [59]—MRI-Based Multiple Instance Convolutional Neural Network for Increased Accuracy in the Differentiation of Borderline and Malignant Epithelial Ovarian Tumours;
***Jian et al. [64]—Multiple instance convolutional neural network with modality-based attention and contextual multi-instance learning pooling layer for effective differentiation between borderline and malignant epithelial ovarian tumours

Back to article page