Skip to main content
Insights into Imaging
Download PDF
  • Original Article
  • Open access
  • Published:

Systematic review with radiomics quality score of cholangiocarcinoma: an EuSoMII Radiomics Auditing Group Initiative

Insights into Imaging volume 14, Article number: 21 (2023) Cite this article

Abstract

Objectives

To systematically review current research applications of radiomics in patients with cholangiocarcinoma and to assess the quality of CT and MRI radiomics studies.

Methods

A systematic search was conducted on PubMed/Medline, Web of Science, and Scopus databases to identify original studies assessing radiomics of cholangiocarcinoma on CT and/or MRI. Three readers with different experience levels independently assessed quality of the studies using the radiomics quality score (RQS). Subgroup analyses were performed according to journal type, year of publication, quartile and impact factor (from the Journal Citation Report database), type of cholangiocarcinoma, imaging modality, and number of patients.

Results

A total of 38 original studies including 6242 patients (median 134 patients) were selected. The median RQS was 9 (corresponding to 25.0% of the total RQS; IQR 1–13) for reader 1, 8 (22.2%, IQR 3–12) for reader 2, and 10 (27.8%; IQR 5–14) for reader 3. The inter-reader agreement was good with an ICC of 0.75 (95% CI 0.62–0.85) for the total RQS. All studies were retrospective and none of them had phantom assessment, imaging at multiple time points, nor performed cost-effectiveness analysis. The RQS was significantly higher in studies published in journals with impact factor > 4 (median 11 vs. 4, p = 0.048 for reader 1) and including more than 100 patients (median 11.5 vs. 0.5, p < 0.001 for reader 1).

Conclusions

Quality of radiomics studies on cholangiocarcinoma is insufficient based on the radiomics quality score. Future research should consider prospective studies with a standardized methodology, validation in multi-institutional external cohorts, and open science data.

Key points

  • The quality of current radiomics studies on cholangiocarcinoma is insufficient, with a median radiomics quality score of 8–10, corresponding to 22–28% of the ideal quality score.

  • None of the current studies conducted phantom assessment, imaging at multiple time points, prospective registration in a trial database, nor cost-effectiveness analysis.

  • The inter-reader agreement of the radiomics quality score is good (ICC of 0.75; 95% CI 0.62–0.85) among readers with different levels of experience.

Introduction

Radiomics is a rapidly expanding area of active research with promising results based on the extraction and analysis of a large number of quantitative features from biomedical images [1]. Recent radiomics studies aimed to construct predictive models that can be combined with qualitative radiological features, clinical characteristics, and laboratory markers to develop decision support tools and improve patients’ care [1]. Several research studies proposed models based on computed tomography (CT) and magnetic resonance imaging (MRI) exams, with high performances for preoperative lesion characterization, prediction of treatment response, and assessment of prognosis after surgical resection [2]. Despite the promising results in research setting, there is still very limited translation in clinical practice due to the limitations of current radiomics research. These include heterogeneity of imaging acquisition protocols, segmentation, type of extracted features, and lack of validation in multicenter setting [3, 4]. Quality of radiomics studies represents a significant landmark for improvement of radiomics research and future clinical applications.

The radiomics quality score (RQS) has been proposed by Lambin et al. [5] for assessing the quality of radiomics studies based on 16 items related to the main steps of radiomics workflow. In the setting of liver imaging, recent systematic reviews have applied the RQS for assessment of quality of radiomics studies on hepatocellular carcinoma [6,7,8,9] and hepatic metastases [10] reporting an overall RQS of 8–14 (corresponding to 23–39% of the total score) and 10 (28%), respectively. Cholangiocarcinoma is the most common malignancy originating from the bile ducts and the second most common primary intrahepatic carcinoma [11]. Cholangiocarcinoma can occur in various location with heterogeneous imaging appearance on CT or MRI, and it is characterized by high biological aggressiveness and poor prognosis [11]. Recently, a growing number of radiomics applications have been proposed in patients with cholangiocarcinoma imaged with either CT or MRI, including differential diagnosis with other hepatic malignancies, prediction of lymph node metastasis, and prediction of recurrence after curative resection. However, to date the quality of radiomics studies in cholangiocarcinoma has not been comprehensively investigated. Assessment of quality in radiomics research studies is a necessary and fundamental step for the improvement of radiomics research and future implementation in clinical practice.

This systematic review aims to provide an overview of the current research applications of radiomics in patients with cholangiocarcinoma and to assess the quality of CT and MRI radiomics studies.

Materials and methods

This study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines [12]. The review protocol was registered on the International Prospective Register of Systematic Reviews (CRD42022295218).

Literature search strategy

A systematic search was conducted to identify studies on PubMed/Medline, Web of Science, and Scopus databases using the following terms: “texture,” “radiomics,” “machine learning,” “artificial intelligence,” “cholangiocarcinoma” and “biliary cancer.” Detailed search strings are reported in the Additional file 1. The literature search was performed for articles published between 01/01/2010 and 30/11/2021.

Eligibility criteria

After removal of duplicate studies, three Authors (R.Ca., F.V., and L.U., radiologists, each with five years of experience in radiomics studies) independently evaluated the titles and abstracts of all studies to exclude ineligible papers according to the following criteria: (1) non-English studies; (2) animal studies; (3) abstracts of conference papers; (4) reviews, systematic reviews, and case reports. The full texts of the relevant articles were read to determine their inclusion. The following eligibility criteria were applied during full-text manuscript review for the inclusion of original papers: (1) radiomics studies based on the evaluation of quantitative features obtained from tumor segmentations of cholangiocarcinoma; (2) features extracted from CT or MRI exams. Studies assessing only semantic features, other lesions than cholangiocarcinoma, or features on other diagnostic exams (i.e., ultrasound or PET/CT due to their limited applications in cholangiocarcinoma) were excluded. Any disagreement between reviewers was resolved with consensus discussion.

Data extraction

The following data were collected from the included studies: authors, journal with its type, journal ranking according to quartile and impact factor, year of publication of the study and country based on the Institutions in which the CT/MRI of the study population exams were acquired. The journals were classified into imaging, clinical, and computer science according to the main journal category of Web of Science. The journal quartile, according to the main journal scientific sector, and impact factor were retrieved from the Journal Citation Report database, and the quartile and impact factor of the year of publication were registered. For articles published in 2021–2022, the 2020 reports were considered as this is the last available at the time of data analysis. The full manuscripts were assessed to collect the following data: type of the study (retrospective or prospective), number of involved Institutions, total number of included patients (divided into training and validation cohorts), type of cholangiocarcinoma (i.e., intrahepatic, perihilar, or extrahepatic), imaging modality (CT and/or MRI), sequences and/or phases in which the segmentation was performed, software used for segmentation and imaging analysis, and number of extracted radiomics features.

The studies were grouped according to the main purpose of investigation: diagnostic (including radiomics analysis for the differential diagnosis among hepatic lesions, prediction of tumor histopathological differentiation and markers, or lymph node involvement), prognostic (prediction of early recurrence and survival), and treatment response (response to locoregional or systemic treatments) studies.

Radiomics quality score assessment

Three different readers from distinct Institutions and with different levels of experience (Reader 1, R1, A.P., a radiologist with 4 years of experience in radiomics research and with experience on the RQS assessment, Reader 2, R2, M.E.K., a radiologist with 10 years of experience in radiological research and 4 years of experience in radiomics research, and Reader 3, R3, E.P., a radiologist with 9 years of experience in radiological research), not involved in manuscript screening, independently evaluated all the studies using the RQS [5]. Before the manuscript assessment, a training session was held to discuss the main items of the RQS and examples on manuscripts not included in this systematic review. Both full-text manuscripts and Supplementary Materials were screened. The RQS consists of 16 items divided by three main checkpoints: the first checkpoint includes item 1, the second includes items from 2 to 4, and the third is composed by items from 5 to 16 [5]. The detailed description of the RQS is available in the Additional file 2: Table S1. The total RQS (ranging from − 8 to + 36) and the percentage of the total score (0–100%) were recorded from all three readers [5].

Statistical analysis

Categorical variables were reported as numbers, proportions, and percentages, while continuous variables were reported as medians and interquartile ranges (IQR), after testing for normal distribution by applying the Shapiro–Wilk normality test. Adherence rate to the reporting quality of the RQS was calculated for the most experience reader (R1), considering the proportion of articles obtaining at least one point in each specific item. Differences in total RQS according to publication and study characteristics were evaluated by using the Kruskal–Wallis or the Mann–Whitney U test, as appropriate. The correlation between total RQS, journal impact factor, and number of included patients was calculated by using the Spearman’s rank correlation coefficient (Spearman’s ρ).

The intraclass correlation coefficient (ICC) with 95% confidence intervals (CI), based on an absolute-agreement with 2-way mixed-effects model, was used to assess the inter-reader agreement in the total and percentage RQS among the three readers. Agreement was categorized as poor (ICC < 0.50), moderate (ICC = 0.50–0.75), good (ICC = 0.75–0.90), or excellent (ICC > 0.90) [13].

A p value < 0.05 was considered to be statistically significant. Statistical analyses were conducted by using the SPSS Software (v26.0. IBM, Armonk, NY, USA).

Results

Literature search

The systematic search initially identified 503 articles (Fig. 1). After removing 214 duplicated manuscripts, 289 were screened by their title and abstracts, and 214 studies underwent full-text screening to assess their eligibility. Finally, 38 original articles on radiomics of cholangiocarcinoma were included for RQS assessment [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51].

Fig. 1
figure 1

Flow diagram of the study selection process

Full size image

Characteristics of the included studies

The characteristics of included publications are summarized in Table 1. Among the included original articles, 18/38 (47.4%) were published in an imaging journal, 15/38 (39.5%) in a clinical journal, and the remaining (5/38, 13.1%) in a computer science journal. Twenty-one (55.3%) articles were published in 2021, 9/38 (23.7%) in 2020, 5/38 (13.2%) in 2019, and 3/38 (7.8%) in 2018 or earlier years. The 17/38 (44.7%) of included articles were published in first quartile journals, with an overall median impact factor of 4.43 (IQR, 3.50–5.31). The study population most frequently originated from China (27/38, 71.1%), followed by the USA (3/38, 7.8%). Thirty (79.0%) studies were performed at one Institution, 7/38 (18.4%) were performed in two Institutions, and only one (2.6%) involved six different Centers. All the included studies were conducted retrospectively.

Table 1 General characteristics of the included studies
Full size table

Study purpose and methodology are detailed in Table 2. The most common study aims (Fig. 2) included differential diagnosis against other hepatic lesions (10/38, 26.3%), prediction of survival after surgical resection (10/38, 26.3%), and prediction of lymph node metastases (7/38, 18.4%). Only one article explored the potential of radiomics for the prediction of therapeutic response to radioembolization in intrahepatic cholangiocarcinoma [23]. The total number of patients was 6242 (median = 134 per study; IQR, 98–198). Intrahepatic, perihilar, and extrahepatic cholangiocarcinoma were assessed in 29/38 (76.3%), 4/38 (10.5%), and 5/38 (13.2%) papers, respectively. CT was the most commonly used imaging technique (20/38, 52.6%), while MRI was adopted in 16/38 (42.1%) studies. Only two (5.3%) used both techniques. Lesion segmentations for radiomics features extraction were more commonly performed in the hepatic arterial phase (27/38, 71.1%) and/or portal venous phase (25/38, 65.8%), almost always by manually drawing the region of interest (35/38, 92.1%). Segmentation of the peritumoral or adjacent hepatic parenchyma was performed in only 5/38 (13.1%) studies.

Table 2 Main purposes and methodology of the included studies
Full size table
Fig. 2
figure 2

Overview of radiomics research purposes. Notably each study could include multiple purposes

Full size image

Radiomics quality score

Results of the total RQS by the three independent readers are summarized in Table 3. Details on the items’ score by each reader are provided in the Additional file 2: Tables S2, S3, and S4. The median RQS was 9 (corresponding to the 25.0% of the total RQS; IQR 1–13) for R1, 8 (22.2%, IQR 3–12) for R2, and 10 (27.8%; IQR 5–14) for R3. The inter-reader agreement for was good with an ICC of 0.75 (95% CI 0.62–0.85) for the total RQS and 0.77 (95% CI 0.65–0.86) for the RQS percentage scores.

Table 3 Total radiomics quality score (RQS) with percentage of the total score of the included studies assessed by three independent readers
Full size table

Adherence rate of each item (according to R1) is illustrated in Fig. 3. For the first checkpoint (item 1), 33/38 (88.6%) studies provided a well-documented image protocol. In the second checkpoint (items from 2 to 4), 26/38 (68.4%) studies had multiple segmentations, but none performed phantom assessment or imaging at multiple time points. In the third checkpoint (items from 5 to 16), feature reduction and adjustment for multiple tests was employed in 30/38 (78.9%) cases, with non-radiomics features included in 22/38 (57.9%) multivariate analyses. Only two (5.3%) articles discussed biological correlates related to the radiomics models. Cutoff analyses, discrimination statistics, and calibration statistics were available in 9/38 (23.7%), 37/38 (97.4%), and 14/38 (36.8%) investigations, respectively. Regarding the validation of the radiomics models, in most studies (23/38, 60.5%) it was based on an internal cohort, but validation was lacking in 13/38 (34.2%) investigations. Comparison with the gold standard and discussion of potential clinical utility were addressed in 20/38 (52.6%) and 11/38 (28.9%) studies, respectively. None of the assessed article was prospectively registered in a trial database or provided cost-effectiveness analysis, and only three (7.9%) made their code or data publicity available.

Fig. 3
figure 3

Adherence rate to the reporting quality of each item included in the radiomics quality score according to the most experienced reader (R1)

Full size image

Subgroup analyses

Results of the subgroup analyses are reported in Table 4. No statistical differences were found according to the type of journal, year of publication, journal quartile, type of included cholangiocarcinoma, and imaging modalities. None of the RQS items (according to R1) was significantly higher in first quartile journals (p ≥ 0.101). Journals with impact factors > 4 published studies with significantly higher RQS according to the R1 (p = 0.048) and R2 (p = 0.035). The RQS was significantly higher in studies including more than 100 patients (p < 0.001 for all the readers).

Table 4 Subgroup analyses of total radiomics quality score assessed by the three independent readers
Full size table

For all the three readers, there was a statistically significant high correlation between the total RQS and number of included patients (p < 0.001 for all the readers) (Table 5). No significant correlation was observed between the total RQS and other characteristics.

Table 5 Correlation between total radiomics quality score assessed by three independent readers, journal impact factor, number of included patients, and number or radiomics features
Full size table

Discussion

Inadequate quality of radiomics studies is emerging as a major issue of current literature, contributing to the slow transition from research to clinical application in this field [52]. This systematic review of 38 radiomics studies on cholangiocarcinoma demonstrates a suboptimal quality of the current publications assessed through the RQS, with an overall total score of 8–10, corresponding to about one quarter of the ideal quality for this type of study. This is in line with other systematic reviews based on the RQS assessing radiomics of hepatic lesions, reporting a median RQS of 8–14 corresponding to 23–39% of the total score [6,7,8,9,10]. Importantly, in this review none of the included studies had phantom assessment, imaging at multiple time points, prospective registration in a trial database, nor performed cost-effectiveness analysis. These items account for 10 points (28%) of the total RQS [5].

Radiomics has been applied as a diagnostic tool for the differential diagnosis between cholangiocarcinoma and other hepatic tumors, for preoperative identification of histopathological and molecular markers associated with poor prognosis, and for predicting postoperative survival, while there is still a very limited experience on therapeutic response and advanced lesions that were not suitable for surgical resection [23]. To date, all studies on radiomics of cholangiocarcinoma are retrospective, mostly based on a single-center dataset with lack of validation cohorts in 34% of them. This is a relevant issue, determining a loss of 5 points in the total RQS, as external validation is a key item prior to clinical implementation of classification models. Only the study by Park et al. [25] validated a radiomics model for the prediction of postoperative outcome in patients with intrahepatic cholangiocarcinoma in an external test dataset from five different institutions (obtaining the maximum score of + 5 points in item 12 of the RQS). This means that even though most studies focused on radiomics of cholangiocarcinoma have a great potential, their results are still confined to the academic centers where the model originated. Further investigations should focus on the validation of existing models in a multicentric context rather than proposing alternative models based on a single-center experiences. Prospective validation of the radiomics models is also needed to evaluate their potential in clinical practice focusing on relevant patients’ outcomes such us evaluation of overall survival after treatment. In this setting, open science data providing the code and radiomics data is of utmost importance to facilitate the widespread application of radiomics and the reproducibility of the proposed models. Nevertheless, less than 10% of studies included in this systematic review made their code or data publicity available.

Despite the need of well-conducted radiomics workflow has been emphasized over the last years, the quality of published radiomics papers on cholangiocarcinoma according to the RQS has not increased when comparing 2021 versus 2013–2020. High-impact journals have provided guidance highlighting the need for robust data and accurate methodology for radiomics research [1, 53, 54]. However, the analysis of current cholangiocarcinoma studies demonstrates no significant difference in the RQS based on the journal type or quartile, even though a tendency of higher RQS was observed in journals with impact factor greater than 4 in two out of three readers. In prior studies, no difference according to journal metrics were found by Spadarella et al. [55] in RQS of nasopharyngeal cancer studies, while Ponsiglione et al. [56] and Chang et al. [57] observed significantly higher RQS in journals with higher impact factor or quartile for cardiac imaging studies, respectively. Therefore, the explosion of research on radiomics, machine learning, and data-based science led to an increased number of published radiomics papers not followed by a significant increase in quality of those studies. This may be related to the overall tendency to perform and publish explorative radiomics studies based on the novelty of the topic rather than to improve the strict methodology and workflow of radiomics analysis [52, 58]. Standardization of the acquisition protocols in liver imaging is a fundamental effort in order to minimize the variability of radiomics features across centers and scanners [59]. Furthermore, the International Biomarker Standardization Initiative (IBSI) is working toward standardization of extraction of quantitative features extracted from medical images and it already provided reference values for radiomics features on CT [60].

The RQS provides a detailed description of each item’s score [5]. However, its application can be affected by the reader’s experience and interpretation of each item according to the data available in the papers. All the studies included in this review were evaluated by three independent readers with different levels of research experience and RQS assessment, which resulted in a good inter-observer agreement. Few studies evaluated the reproducibility of the RQS with discordant results (reported ICC between 0.57 and 0.99) and, to our knowledge, none of these studies evaluated the inter-observed agreement in readers from different Institutions [55, 61,62,63]. It should be noted that the RQS is based on expert opinion and currently not endorsed by scientific societies and it is limited by strong dependence on the methodological quality of the ideal radiomics workflow with low relevance to the potential clinical impact. Some of the items, such as phantom assessment and imaging at multiple time points, remain difficult to be investigated when considering real-word data based on retrospective observational studies. Nevertheless, the application of this score could be encouraged for the quality assessment of the papers submitted to peer-review journals in order to facilitate manuscript decisions and improve the overall quality of radiomics studies.

Some limitations pertain to this study. First, a meta-analysis was not performed due to the heterogeneity of the included studies, with a relatively small number of papers assessing the radiomics models for a specific aim, which makes challenging to pool data for a strong meta-analysis. Secondly, cholangiocarcinomas are rarer tumors compared to hepatocellular carcinoma and hepatic metastasis, and the applications of the radiomics in this field is relatively new. This is demonstrated by the fact that 55% of the included studies were published in 2021. Finally, papers with radiomics applied to cholangiocarcinoma on ultrasound and PET/CT were not included due to the limited clinical applicability in patients with cholangiocarcinoma and highly exploratory nature of radiomics analyses with these imaging modalities.

In conclusion, radiomics studies on cholangiocarcinoma demonstrated an insufficient quality with a low total radiomics quality score. Further prospective studies are needed with a standardized methodology, validation in multi-imitational external cohorts, and open science data in order to translate the promising research results in the field of radiomics into useful applications to improved patients’ management in many clinical scenarios.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CI:

Confidence interval

CT:

Computed tomography

IBSI:

International Biomarker Standardization Initiative

ICC:

Intraclass correlation coefficient

IQR:

Interquartile range

MRI:

Magnetic resonance imaging

RQS:

Radiomics quality score

References

  1. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577. https://doi.org/10.1148/radiol.2015151169

    Article  Google Scholar 

  2. Lubner MG, Smith AD, Sandrasegaran K, Sahani DV, Pickhardt PJ (2017) CT texture analysis: definitions, applications, biologic correlates, and challenges. Radiographics 37:1483–1503. https://doi.org/10.1148/rg.2017170056

    Article  Google Scholar 

  3. Ursprung S, Beer L, Bruining A et al (2020) Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma-a systematic review and meta-analysis. Eur Radiol 30:3558–3566. https://doi.org/10.1007/s00330-020-06666-3

    Article  Google Scholar 

  4. Borhani AA, Catania R, Velichko YS, Hectors S, Taouli B, Lewis S (2021) Radiomics of hepatocellular carcinoma: promising roles in patient selection, prediction, and assessment of treatment response. Abdom Radiol (NY) 46:3674–3685. https://doi.org/10.1007/s00261-021-03085-w

    Article  Google Scholar 

  5. Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762. https://doi.org/10.1038/nrclinonc.2017.141

    Article  Google Scholar 

  6. Wakabayashi T, Ouhmich F, Gonzalez-Cabrera C et al (2019) Radiomics in hepatocellular carcinoma: a quantitative review. Hepatol Int 13:546–559. https://doi.org/10.1007/s12072-019-09973-0

    Article  Google Scholar 

  7. Harding-Theobald E, Louissaint J, Maraj B et al (2021) Systematic review: radiomics for the diagnosis and prognosis of hepatocellular carcinoma. Aliment Pharmacol Ther 54:890–901. https://doi.org/10.1111/apt.16563

    Article  Google Scholar 

  8. Wang Q, Li C, Zhang J et al (2021) Radiomics models for predicting microvascular invasion in hepatocellular carcinoma: a systematic review and radiomics quality score assessment. Cancers (Basel) 13:5864. https://doi.org/10.3390/cancers13225864

    Article  Google Scholar 

  9. Zhong X, Long H, Su L et al (2022) Radiomics models for preoperative prediction of microvascular invasion in hepatocellular carcinoma: a systematic review and meta-analysis. Abdom Radiol (NY) 47:2071–2088. https://doi.org/10.1007/s00261-022-03496-3

    Article  Google Scholar 

  10. Fiz F, Viganò L, Gennaro N et al (2020) Radiomics of liver metastases: a systematic review. Cancers (Basel) 12:2881. https://doi.org/10.3390/cancers12102881

    Article  CAS  Google Scholar 

  11. Joo I, Lee JM, Yoon JH (2018) Imaging diagnosis of intrahepatic and perihilar cholangiocarcinoma: recent advances and challenges. Radiology 288:7–13. https://doi.org/10.1148/radiol.2018171187

    Article  Google Scholar 

  12. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:b2535. https://doi.org/10.1136/bmj.b2535

    Article  Google Scholar 

  13. Benchoufi M, Matzner-Lober E, Molinari N, Jannot AS, Soyer P (2020) Interobserver agreement issues in radiology. Diagn Interv Imaging 101:639–641. https://doi.org/10.1016/j.diii.2020.09.001

    Article  CAS  Google Scholar 

  14. Chu H, Liu Z, Liang W et al (2021) Radiomics using CT images for preoperative prediction of futile resection in intrahepatic cholangiocarcinoma. Eur Radiol 31:2368–2376. https://doi.org/10.1007/s00330-020-07250-5

    Article  Google Scholar 

  15. Duda D, Krętowski M, Bézy-Wendling J (2013) Computer-aided diagnosis of liver tumors based on multi-image texture analysis of contrast-enhanced CT. Selection of the most appropriate texture features. Stud Log Gramm Rhetor 35:49–70. https://doi.org/10.2478/slgr-2013-0039

    Article  Google Scholar 

  16. Hamm CA, Wang CJ, Savic LJ et al (2019) Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol 29:3338–3347. https://doi.org/10.1007/s00330-019-06205-9

    Article  Google Scholar 

  17. Huang X, Shu J, Yan Y et al (2021) Feasibility of magnetic resonance imaging-based radiomics features for preoperative prediction of extrahepatic cholangiocarcinoma stage. Eur J Cancer 155:227–235. https://doi.org/10.1016/j.ejca.2021.06.053

    Article  Google Scholar 

  18. Ji GW, Zhu FP, Zhang YD et al (2019) A radiomics approach to predict lymph node metastasis and clinical outcome of intrahepatic cholangiocarcinoma. Eur Radiol 29:3725–3735. https://doi.org/10.1007/s00330-019-06142-7

    Article  Google Scholar 

  19. Ji GW, Zhang YD, Zhang H et al (2019) Biliary tract cancer at CT: a radiomics-based model to predict lymph node metastasis and survival outcomes. Radiology 290:90–98. https://doi.org/10.1148/radiol.2018181408

    Article  Google Scholar 

  20. King MJ, Hectors S, Lee KM et al (2020) Outcomes assessment in intrahepatic cholangiocarcinoma using qualitative and quantitative imaging features. Cancer Imaging 20:43. https://doi.org/10.1186/s40644-020-00323-0

    Article  Google Scholar 

  21. Liang W, Xu L, Yang P et al (2018) Novel nomogram for preoperative prediction of early recurrence in intrahepatic cholangiocarcinoma. Front Oncol 8:360. https://doi.org/10.3389/fonc.2018.00360

    Article  Google Scholar 

  22. Liu X, Khalvati F, Namdar K et al (2021) Can machine learning radiomics provide pre-operative differentiation of combined hepatocellular cholangiocarcinoma from hepatocellular carcinoma and cholangiocarcinoma to inform optimal treatment planning? Eur Radiol 31:244–255. https://doi.org/10.1007/s00330-020-07119-7

    Article  CAS  Google Scholar 

  23. Mosconi C, Cucchetti A, Bruno A et al (2020) Radiomics of cholangiocarcinoma on pretreatment CT can identify patients who would best respond to radioembolisation. Eur Radiol 30:4534–4544. https://doi.org/10.1007/s00330-020-06795-9

    Article  Google Scholar 

  24. Nakai H, Fujimoto K, Yamashita R et al (2021) Convolutional neural network for classifying primary liver cancer based on triple-phase CT and tumor marker information: a pilot study. Jpn J Radiol 39:690–702. https://doi.org/10.1007/s11604-021-01106-8

    Article  CAS  Google Scholar 

  25. Park HJ, Park B, Park SY et al (2021) Preoperative prediction of postsurgical outcomes in mass-forming intrahepatic cholangiocarcinoma based on clinical, radiologic, and radiomics features. Eur Radiol 31:8638–8648. https://doi.org/10.1007/s00330-021-07926-6

    Article  Google Scholar 

  26. Park S, Lee JM, Park J et al (2021) Volumetric CT texture analysis of intrahepatic mass-forming cholangiocarcinoma for the prediction of postoperative outcomes: fully automatic tumor segmentation versus semi-automatic segmentation. Korean J Radiol 22:1797–1808. https://doi.org/10.3348/kjr.2021.0055

    Article  Google Scholar 

  27. Ponnoprat D, Inkeaw P, Chaijaruwanich J et al (2020) Classification of hepatocellular carcinoma and intrahepatic cholangiocarcinoma based on multi-phase CT scans. Med Biol Eng Comput 58:2497–2515. https://doi.org/10.1007/s11517-020-02229-2

    Article  Google Scholar 

  28. Qin H, Hu X, Zhang J et al (2021) Machine-learning radiomics to predict early recurrence in perihilar cholangiocarcinoma after curative resection. Liver Int 41:837–850. https://doi.org/10.1111/liv.14763

    Article  Google Scholar 

  29. Sadot E, Simpson AL, Do RK et al (2015) Cholangiocarcinoma: correlation between molecular profiling and imaging phenotypes. PLoS One 10:e0132953. https://doi.org/10.1371/journal.pone.0132953

    Article  CAS  Google Scholar 

  30. Silva M, Maddalo M, Leoni E et al (2021) Integrated prognostication of intrahepatic cholangiocarcinoma by contrast-enhanced computed tomography: the adjunct yield of radiomics. Abdom Radiol (NY) 46:4689–4700. https://doi.org/10.1007/s00261-021-03183-9

    Article  Google Scholar 

  31. Tang Y, Yang CM, Su S, Wang WJ, Fan LP, Shu J (2021) Machine learning-based Radiomics analysis for differentiation degree and lymphatic node metastasis of extrahepatic cholangiocarcinoma. BMC Cancer 21:1268. https://doi.org/10.1186/s12885-021-08947-6

    Article  CAS  Google Scholar 

  32. Tang Y, Zhang T, Zhou X et al (2021) The preoperative prognostic value of the radiomics nomogram based on CT combined with machine learning in patients with intrahepatic cholangiocarcinoma. World J Surg Oncol 19:45. https://doi.org/10.1186/s12957-021-02162-0

    Article  Google Scholar 

  33. Wang X, Wang S, Yin X, Zheng Y (2022) MRI-based radiomics distinguish different pathological types of hepatocellular carcinoma. Comput Biol Med 141:105058. https://doi.org/10.1016/j.compbiomed.2021.105058

    Article  CAS  Google Scholar 

  34. Wang Y, Shao J, Wang P et al (2021) Deep learning radiomics to predict regional lymph node staging for hilar cholangiocarcinoma. Front Oncol 11:721460. https://doi.org/10.3389/fonc.2021.721460

    Article  Google Scholar 

  35. Xu H, Zou X, Zhao Y et al (2021) Differentiation of intrahepatic cholangiocarcinoma and hepatic lymphoma based on radiomics and machine learning in contrast-enhanced computer tomography. Technol Cancer Res Treat 20:15330338211039124. https://doi.org/10.1177/15330338211039125

    Article  CAS  Google Scholar 

  36. Xu L, Wan Y, Luo C et al (2021) Integrating intratumoral and peritumoral features to predict tumor recurrence in intrahepatic cholangiocarcinoma. Phys Med Biol. https://doi.org/10.1088/1361-6560/ac01f3

    Article  Google Scholar 

  37. Xu L, Yang P, Liang W et al (2019) A radiomics approach based on support vector machine using MR images for preoperative lymph node status evaluation in intrahepatic cholangiocarcinoma. Theranostics 9:5374–5385. https://doi.org/10.7150/thno.34149

    Article  Google Scholar 

  38. Xue B, Wu S, Zheng M et al (2021) Development and validation of a Radiomic-based model for prediction of intrahepatic Cholangiocarcinoma in patients with intrahepatic Lithiasis complicated by imagologically diagnosed mass. Front Oncol 10:598253. https://doi.org/10.3389/fonc.2020.598253

    Article  Google Scholar 

  39. Xue B, Wu S, Zhang M et al (2021) A radiomic-based model of different contrast-enhanced CT phase for differentiate intrahepatic cholangiocarcinoma from inflammatory mass with hepatolithiasis. Abdom Radiol (NY) 46:3835–3844. https://doi.org/10.1007/s00261-021-03027-6

    Article  Google Scholar 

  40. Yang C, Huang M, Li S et al (2020) Radiomics model of magnetic resonance imaging for predicting pathological grading and lymph node metastases of extrahepatic cholangiocarcinoma. Cancer Lett 470:1–7. https://doi.org/10.1016/j.canlet.2019.11.036

    Article  CAS  Google Scholar 

  41. Yao X, Huang X, Yang C et al (2020) A novel approach to assessing differentiation degree and lymph node metastasis of extrahepatic cholangiocarcinoma: prediction using a radiomics-based particle swarm optimization and support vector machine model. JMIR Med Inform 8:e23578. https://doi.org/10.2196/23578

    Article  Google Scholar 

  42. Zhang J, Wu Z, Zhang X et al (2020) Machine learning: an approach to preoperatively predict PD-1/PD-L1 expression and outcome in intrahepatic cholangiocarcinoma using MRI biomarkers. ESMO Open 5:e000910. https://doi.org/10.1136/esmoopen-2020-000910

    Article  Google Scholar 

  43. Zhang J, Wang X, Zhang L et al (2020) Radiomics predict postoperative survival of patients with primary liver cancer with different pathological types. Ann Transl Med 8:820. https://doi.org/10.21037/atm-19-4668

    Article  CAS  Google Scholar 

  44. Zhang J, Huang Z, Cao L et al (2020) Differentiation combined hepatocellular and cholangiocarcinoma from intrahepatic cholangiocarcinoma based on radiomics machine learning. Ann Transl Med 8:119. https://doi.org/10.21037/atm.2020.01.126

    Article  CAS  Google Scholar 

  45. Zhang J, Wu Z, Zhao J et al (2021) Intrahepatic cholangiocarcinoma: MRI texture signature as predictive biomarkers of immunophenotyping and survival. Eur Radiol 31:3661–3672. https://doi.org/10.1007/s00330-020-07524-y

    Article  CAS  Google Scholar 

  46. Zhao J, Zhang W, Fan CL et al (2021) Development and validation of preoperative magnetic resonance imaging-based survival predictive nomograms for patients with perihilar cholangiocarcinoma after radical resection: a pilot study. Eur J Radiol 138:109631. https://doi.org/10.1016/j.ejrad.2021.109631

    Article  Google Scholar 

  47. Zhao J, Zhang W, Zhu YY et al (2022) Development and validation of noninvasive MRI-based signature for preoperative prediction of early recurrence in perihilar cholangiocarcinoma. J Magn Reson Imaging 55:787–802. https://doi.org/10.1002/jmri.27846

    Article  Google Scholar 

  48. Zhao L, Ma X, Liang M et al (2019) Prediction for early recurrence of intrahepatic mass-forming cholangiocarcinoma: quantitative magnetic resonance imaging combined with prognostic immunohistochemical markers. Cancer Imaging 19:49. https://doi.org/10.1186/s40644-019-0234-4

    Article  Google Scholar 

  49. Zhou Y, Zhou G, Zhang J, Xu C, Wang X, Xu P (2021) Radiomics signature on dynamic contrast-enhanced MR images: a potential imaging biomarker for prediction of microvascular invasion in mass-forming intrahepatic cholangiocarcinoma. Eur Radiol 31:6846–6855. https://doi.org/10.1007/s00330-021-07793-1

    Article  Google Scholar 

  50. Zhu Y, Mao Y, Chen J et al (2021) Value of contrast-enhanced CT texture analysis in predicting IDH mutation status of intrahepatic cholangiocarcinoma. Sci Rep 11:6933. https://doi.org/10.1038/s41598-021-86497-4

    Article  CAS  Google Scholar 

  51. Zhu Y, Mao Y, Chen J et al (2021) Radiomics-based model for predicting early recurrence of intrahepatic mass-forming cholangiocarcinoma after curative tumor resection. Sci Rep 11:18347. https://doi.org/10.1038/s41598-021-97796-1

    Article  CAS  Google Scholar 

  52. Pinto Dos Santos D, Dietzel M, Baessler B (2021) A decade of radiomics research: Are images really data or just patterns in the noise? Eur Radiol 31:1–4

    Article  Google Scholar 

  53. van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B (2020) Radiomics in medical imaging-“how-to” guide and critical reflection. Insights Imaging 11:91. https://doi.org/10.1186/s13244-020-00887-2

    Article  Google Scholar 

  54. Shur JD, Doran SJ, Kumar S et al (2021) Radiomics in oncology: a practical guide. Radiographics 41:1717–1732. https://doi.org/10.1148/rg.2021210037

    Article  Google Scholar 

  55. Spadarella G, Calareso G, Garanzini E, Ugga L, Cuocolo A, Cuocolo R (2021) MRI based radiomics in nasopharyngeal cancer: systematic review and perspectives using radiomic quality score (RQS) assessment. Eur J Radiol 140:109744. https://doi.org/10.1016/j.ejrad.2021.109744

    Article  Google Scholar 

  56. Ponsiglione A, Stanzione A, Cuocolo R et al (2022) Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment. Eur Radiol 32:2629–2638. https://doi.org/10.1007/s00330-021-08375-x

    Article  Google Scholar 

  57. Chang S, Han K, Suh YJ, Choi BW (2022) Quality of science and reporting for radiomics in cardiac magnetic resonance imaging studies: a systematic review. Eur Radiol. https://doi.org/10.1007/s00330-022-08587-9

    Article  Google Scholar 

  58. Pinto Dos Santos D (2022) Radiomics in endometrial cancer and beyond-a perspective from the editors of the EJR. Eur J Radiol 150:110266

    Article  Google Scholar 

  59. Carbonell G, Kennedy P, Bane O et al (2022) Precision of MRI radiomics features in the liver and hepatocellular carcinoma. Eur Radiol 32:2030–2040. https://doi.org/10.1007/s00330-021-08282-1

    Article  CAS  Google Scholar 

  60. Zwanenburg A, Vallières M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295:328–338. https://doi.org/10.1148/radiol.2020191145

    Article  Google Scholar 

  61. Stanzione A, Gambardella M, Cuocolo R, Ponsiglione A, Romeo V, Imbriaco M (2020) Prostate MRI radiomics: a systematic review and radiomic quality score assessment. Eur J Radiol 129:109095. https://doi.org/10.1016/j.ejrad.2020.109095

    Article  Google Scholar 

  62. Abdurixiti M, Nijiati M, Shen R, Ya Q, Abuduxiku N, Nijiati M (2021) Current progress and quality of radiomic studies for predicting EGFR mutation in patients with non-small cell lung cancer using PET/CT images: a systematic review. Br J Radiol 94:20201272

    Article  Google Scholar 

  63. Ugga L, Perillo T, Cuocolo R et al (2021) Meningioma MRI radiomics and machine learning: systematic review, quality score assessment, and meta-analysis. Neuroradiology 63:1293–1304. https://doi.org/10.1007/s00234-021-02668-0

    Article  Google Scholar 

Download references

Acknowledgements

The European Society of Medical Imaging Informatics (EuSoMII) supports the Radiomics Auditing Group Initiative with the aim to evaluate and improve the quality of radiomics studies.

Funding

None.

Author information

Authors and Affiliations

  1. Section of Radiology - Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University Hospital “Paolo Giaccone”, Via del Vespro 129, 90127, Palermo, Italy

    Roberto Cannella

  2. Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties (PROMISE), University of Palermo, Via del Vespro, 129, 90127, Palermo, Italy

    Roberto Cannella

  3. Department of Radiology, University Hospital of Padova, Via Nicolò Giustiniani 2, 35128, Padua, Italy

    Federica Vernuccio

  4. Department of Medical Imaging, University Hospital of Heraklion, 71110, Voutes, Crete, Greece

    Michail E. Klontzas

  5. Department of Radiology, School of Medicine, University of Crete, 71003, Heraklion, Crete, Greece

    Michail E. Klontzas

  6. Computational Biomedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology, Vassilika Vouton, 70013, Crete, Greece

    Michail E. Klontzas

  7. Department of Advanced Biomedical Sciences, University of Naples “Federico II”, Via Sergio Pansini 5, 80131, Naples, Italy

    Andrea Ponsiglione & Lorenzo Ugga

  8. Radiology Department Research Institute of Children’s Oncology and Hematology, FSBI “National Medical Research Center of Oncology n.a. N.N. Blokhin” of Ministry of Health of RF, Kashirskoye Highway 24, Moscow, Russia

    Ekaterina Petrash

  9. IRA-Labs, Medical Department, Skolkovo, Bolshoi Boulevard, 30, Building 1, Moscow, Russia

    Ekaterina Petrash

  10. Department of Diagnostic and Interventional Radiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Kerpener Str. 62, 50937, Cologne, Germany

    Daniel Pinto dos Santos

  11. Department of Radiology, University Hospital Frankfurt, Theodor-Stern-Kai 7, 60590, Frankfurt, Germany

    Daniel Pinto dos Santos

  12. Department of Medicine, Surgery, and Dentistry, University of Salerno, Via Salvador Allende 43, 84081, Baronissi, SA, Italy

    Renato Cuocolo

  13. Augmented Reality for Health Monitoring Laboratory (ARHeMLab), Department of Electrical Engineering and Information Technology, University of Naples “Federico II”, Via Sergio Pansini 5, 80131, Naples, Italy

    Renato Cuocolo

Authors
  1. Roberto Cannella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Federica Vernuccio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michail E. Klontzas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea Ponsiglione
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ekaterina Petrash
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lorenzo Ugga
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Daniel Pinto dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Renato Cuocolo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RC (Roberto Cannella) and RC (Renato Cuocolo) contributed to study concept and design. RC (Roberto Cannella), FV, and LU performed literature search, manuscript selection, and data extraction. MEK, AP, and EP provided the radiomics quality score assessment. RC (Roberto Cannella) performed the data analysis and wrote the first manuscript draft. RC (Renato Cuocolo) and DPDS contributed to the results interpretation and provided important intellectual content. All authors reviewed and edited the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Roberto Cannella.

Ethics declarations

Ethics approval and consent to participate:

Not applicable.

Consent for publication

Not applicable.

Competing interests

Roberto Cannella: support for attending meeting from Bracco and Bayer; co-funding by the European Union-FESR or FSE, PON Research and Innovation 2014–2020—DM 1062/2021; is a member of the Insights into Imaging Editorial Board. He has not taken part in the review or selection process of this article. Federica Vernuccio: received support from GE Healthcare to attend a meeting and served as speaker for Guerbet. Andrea Ponsiglione: is a member of the Insights into Imaging Editorial Board. He has not taken part in the review or selection process of this article. Daniel Pinto dos Santos: is a member of the Insights into Imaging Editorial Board, serves as Junior Deputy Editor of European Radiology and received consulting fees from cook medical. He has not taken part in the review or selection process of this article. Renato Cuocolo: serves as an editorial board member of European Radiology and European Radiology Experimental. He has not taken part in the review or selection process of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Detailed search strategy.

Additional file 2. Table S1:

Detailed checklist of the radiomics quality score with corresponding checkpoints and items as reported by Lambin et al [5]. Table S2: Radiomics quality score of the included studies assessed by the Reader 1. Table S3: Radiomics quality score of the included studies assessed by the Reader 2. Table S4: Radiomics quality score of the included studies assessed by the Reader 3.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cannella, R., Vernuccio, F., Klontzas, M.E. et al. Systematic review with radiomics quality score of cholangiocarcinoma: an EuSoMII Radiomics Auditing Group Initiative. Insights Imaging 14, 21 (2023). https://doi.org/10.1186/s13244-023-01365-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13244-023-01365-1

Keywords