Skip to main content
Insights into Imaging
Download PDF

Diagnostic performance of standardized typical CT findings for COVID-19: a systematic review and meta-analysis

Insights into Imaging volume 14, Article number: 96 (2023) Cite this article

Abstract

Objective

To meta-analyze diagnostic performance measures of standardized typical CT findings for COVID-19 and examine these measures by region and national income.

Methods

MEDLINE and Embase were searched from January 2020 to April 2022 for diagnostic studies using the Radiological Society of North America (RSNA) classification or the COVID-19 Reporting and Data System (CO-RADS) for COVID-19. Patient and study characteristics were extracted. We pooled the diagnostic performance of typical CT findings in the RSNA and CO-RADS systems and interobserver agreement. Meta-regression was performed to examine the effect of potential explanatory factors on the diagnostic performance of the typical CT findings.

Results

We included 42 diagnostic performance studies with 6777 PCR-positive and 9955 PCR-negative patients from 18 developing and 24 developed countries covering the Americas, Europe, Asia, and Africa. The pooled sensitivity was 70% (95% confidence interval [CI]: 65%, 74%; I2 = 92%), and the pooled specificity was 90% (95% CI 86%, 93%; I2 = 94%) for the typical CT findings of COVID-19. The sensitivity and specificity of the typical CT findings did not differ significantly by national income and the region of the study (p > 0.1, respectively). The pooled interobserver agreement from 19 studies was 0.72 (95% CI 0.63, 0.81; I2 = 99%) for the typical CT findings and 0.67 (95% CI 0.61, 0.74; I2 = 99%) for the overall CT classifications.

Conclusion

The standardized typical CT findings for COVID-19 provided moderate sensitivity and high specificity globally, regardless of region and national income, and were highly reproducible between radiologists.

Critical relevance statement

Standardized typical CT findings for COVID-19 provided a reproducible high diagnostic accuracy globally.

Key points

  • Standardized typical CT findings for COVID-19 provide high sensitivity and specificity.

  • Typical CT findings show high diagnosability regardless of region or income.

  • The interobserver agreement for typical findings of COVID-19 is substantial.

Graphical abstract

Introduction

The ongoing COVID-19 pandemic, caused by the novel virus SARS-CoV-2, has affected all countries worldwide [1]. As of September 26, 2022, the World Health Organization had reported more than 600 million confirmed cases of COVID-19 and 6.5 million COVID-19-related deaths (World Health Organization dashboard, accessed September 26, 2022). During the pandemic, early and accurate diagnoses of COVID-19 have been crucial for preventing the spread of infection and managing patients promptly.

The reference standard for COVID-19 has been the reverse-transcription polymerase chain reaction (RT-PCR) test using respiratory tract specimens. Despite the high diagnostic accuracy of RT-PCR, it was reported that 20–67% of infected patients had false-negative results, and turnaround times ranged from 6 h to over 48 h in the early pandemic [2]. Moreover, RT-PCR could not differentiate between infectious and non-infectious SARS-CoV-2 particles [3, 4]. The rapid antigen test (RAT) provided results faster than RT-PCR and enabled rapid point-of-care triage [5, 6], but RAT was not available in the early stages of the pandemic.

Chest CT imaging has been widely used for COVID-19 since the initial stages of the pandemic of COVID-19. A meta-analysis of early diagnostic CT studies in April 2020 showed that unstandardized CT interpretation was sensitive but nonspecific to COVID-19 [7]. Subsequently, two major CT classification systems for COVID-19 were proposed for standardized interpretation according to the typicality of CT findings: the four-category Radiological Society of North America (RSNA) classification system [8] and the five-category COVID-19 Reporting and Data System (CO-RADS) [9]. The RSNA and CO-RADS classification systems exhibit a high degree of similarity. Specifically, CO-RADS categories 1, 2, 3–4, and 5 correspond, respectively, to the negative, atypical, indeterminate, and typical categories of the RSNA classification system. Early analysis including nine studies, mostly from developed European countries, showed the possibility that each standardized system might better diagnostic performance than unstandardized CT interpretation [10]. Nevertheless, radiology human and facility resources for COVID-19 differed across countries [11], calling into question whether this standardized interpretation of typicality worked similarly worldwide.

This study aimed to meta-analyze diagnostic performance measures of the standardized typical CT findings for COVID-19 and examine these measures by region and national income.

Materials and methods

We conducted this systematic review and meta-analysis according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [12].

Search strategy

A search of the OVID-MEDLINE and Embase databases was conducted for publications on the diagnostic performance of the RSNA classification and the CO-RADS systems in patients with COVID-19 infection. The following keywords were used in different combinations: (“Radiological Society of North America” OR “RSNA” OR “CO-RADS” OR “CORADS”) AND (“Corona” OR “Coronavirus” OR “COVID-19” OR “SARS-Cov-2” OR “2019nCoV”). The search was restricted to human subjects and English-language studies. In addition, publications that cited the original RSNA classification system [8] and CO-RADS [9] for reporting COVID-19 pneumonia were also searched using the “cited reference” function in OVID- MEDLINE and Embase. The search was updated until April 6, 2022.

Study selection

Original studies were eligible for inclusion if they provided data on the diagnostic performance of the RSNA classification or CO-RADS system in evaluating patients with clinically suspected COVID-19 infection, using RT-PCR as the reference standard. Among them, we included only studies from which diagnostic performance measures (sensitivity and specificity) or the interobserver agreement of the CT classification (relative to RT-PCR) could be extracted. The exclusion criteria were: (1) case reports, review articles, editorials, letters, comments, and conference proceedings; (2) studies with insufficient data to compose a 2-by-2 contingency table to calculate sensitivity and specificity on per-patient level for either the RSNA classification or CO-RADS system; and (3) studies that only provided data on the performance of an artificial intelligence-based analysis. The full texts of the articles were reviewed after selecting potentially eligible abstracts.

Study data extraction

Two of the authors (B.D.N. and S.H.Y.) with 11 and 18 years of clinical experience, respectively, independently extracted the data using a standardized form: (1) patient characteristics; (2) study characteristics; (3) the results of each diagnostic test regarding the results of RT-PCR assays; and (4) the interobserver agreement for the RSNA classification and the CO-RADS system. The two systems identically defined typical CT findings for COVID-19 with different labels as a typical appearance in the RSNA system and grade 5 in the CO-RADS system [9]. Accordingly, we regarded the typical appearance in the RSNA and grade 5 in the CO-RADS system as standardized typical CT findings for COVID-19.

National income was subcategorized into low-income, lower-middle-income, upper-middle-income, and high-income economies based on World Bank data [13]. Developing countries were defined as low- to upper-middle-income economies, and developed countries were defined as high-income economies. Country-level rates of COVID-19 vaccination were sourced from the Our World in Data [14]. Full vaccination was defined as the completion of two or more doses.

Definition of outcomes

The primary outcome of this meta-analysis was the pooled diagnostic performance of typical CT findings (“typical” in the RSNA CT classification or CO-RADS 5). The RSNA and CO-RADS CT classification systems are similar and our study analyzed that CO-RADS categories 1, 2, 3–4, and 5 were interpreted respectively as negative, atypical, indeterminate, and typical categories of the RSNA system, in accordance with previous research [10]. To assist with interpretation, representative CT images were included in the Appendix (Additional file 1: Fig. S1). Secondary outcomes comprised the interobserver agreement for standardized CT classification systems.

Fig. 1
figure 1

Flow diagram of study selection

Full size image

Study quality assessment

The quality of included studies was assessed by two of the authors (B.D.N. and S.H.Y.) using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool, which comprises four key items: patient selection, index test, reference standard, and flow and timing [15].

Statistical analysis

The meta-analysis of diagnostic accuracy was performed using a bivariate generalized linear mixed model, which models sensitivity and specificity jointly and yields unbiased estimates with sparse data [16]. Summary values for sensitivity and specificity, a 95% confidence region for the summary values, and a 95% prediction region were estimated, and the summary receiver-operating characteristic (ROC) curve was derived from the bivariate model. The I2 statistic was used to assess heterogeneity across the studies. The source of between-study variability was explored by including study-level characteristics in the model, and the characteristics were CT classification system, prevalence, mean age, sex (proportion of male subjects), study size, national income, regions (continent), and full vaccination rate. A sensitivity analysis was conducted for studies with more than 50 subjects in both COVID-19 cases and non-COVID-19 patients. The positive predictive value (PPV) and negative predictive value (NPV) of the typical CT findings were estimated and compared to those of RAT [17]: the pooled sensitivity of was 69% (95% confidence interval [CI]: 68%, 70%; I2 = 96%), and the pooled specificity was 99% (95% CI 99%, 99%; I2 = 94%).

In addition, a meta-analysis of observer agreement was conducted for binary classification between typical versus non-typical findings and overall classification. The risk of publication bias in studies reporting diagnostic accuracies and interrater agreement was assessed using Deeks’ regression test. The analysis was done using the lme, altmeta, and metafor packages in R (version 4.1.2).

Results

Literature search

Our literature search process is outlined in Fig. 1. In total, 1,224 articles were screened after the removal of duplicate articles. Of these 1224 articles, 1119 were excluded based on their titles and abstracts. Sixty-three additional articles were excluded after reviewing their full texts, resulting in a total of 46 articles that were finally included: 42 studies for assessing diagnostic accuracy and 19 studies for assessing the interobserver agreement [9, 18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64].

Baseline characteristics

The characteristics of the included studies are outlined in Table 1. Among 42 studies for assessing diagnostic accuracy, the study population inclusion period ranged from January 2020 to August 2021. The total number of the included patients was 16,732 from 16 countries, of whom 6777 had positive RT-PCR and 9,955 had negative RT-PCR results (median patient number, 229; interquartile range, 109 to 526 patients). The median and interquartile range of COVID-19 prevalence were 45.5% and 31.9% to 61.2%, respectively. Thirty-nine studies were conducted when vaccination was not available. There were five prospective studies, and most were retrospective studies. The mean or median age of patients ranged from 40 to 76 years old. There were 18 studies from developing countries (4 studies from lower-middle-income and 14 studies from upper-middle-income countries) [18, 19, 22, 23, 30,31,32, 35, 37, 38, 43,44,45, 47, 48, 50, 51, 58] and 24 studies from developed countries (high income) [9, 20, 21, 24,25,26,27,28,29, 33, 34, 36, 39,40,41,42, 46, 49, 52,53,54,55,56,57]. The included studies were predominantly from Europe (24 studies) [9, 20, 21, 23,24,25,26,27, 29, 31, 32, 34, 35, 37, 38, 41, 42, 47,48,49, 52, 55,56,57], followed by the Americas (10 studies), Asia (seven studies), and Africa (one study). Nineteen studies for assessing the interobserver agreement are summarized in Additional file 1: Table 1.

Table 1 Study characteristics reporting diagnostic accuracy of typical CT findings
Full size table

Quality assessment

The included studies for reporting diagnostic accuracy and interobserver agreement had a relatively low risk of bias in flow and timing, reference standard, and patient selection (Additional file 1: Fig. 2). In regard to the index test, five of 48 studies [18, 19, 25, 59, 63] did not perform RT-PCR result blinding during the interpretation of CT images, causing a high risk of bias. Fifteen of 48 studies [29, 31,32,33,34, 42, 43, 46, 47, 49, 50, 55, 56, 58, 60] lacked a description of blinding regarding the RT-PCR results, for which reason the risk of bias was deemed unclear in those studies.

Fig. 2
figure 2

A Coupled forest plot for the diagnostic accuracy of typical CT findings. B Summary ROC curve in studies reporting both sensitivity and specificity

Full size image

Diagnostic performance of typical CT findings of COVID-19

The pooled sensitivity was 70% (95% CI 65%, 74%; I2 = 92%) and the pooled specificity was 90% (95% CI 86%, 93%; I2 = 94%) for the typical CT findings of COVID-19 (Fig. 2). There was high heterogeneity for both sensitivity and specificity (proportion of prediction region to the area under the ROC curve = 35.8%). The pooled prevalence of COVID-19 was 44.5% (95% CI 37.4%, 51.8%; I2 = 99%).

Meta-regression analyses for the sensitivity and specificity of the typical CT findings (Table 2) revealed that as the prevalence rate increased, the sensitivity tended to increase (p 0.05), and the specificity significantly decreased (p 0.01) (Additional file 1: Fig. 3). The sensitivity of the typical CT findings for COVID-19 did not differ by national income (developing countries, 68% (95% CI 61%, 75%); developed countries, 71% (95% CI 65%, 77%; p 0.57) and the region of the study (p 0.25). There was no significant difference in the specificity of the typical CT findings for COVID-19 by national income (developing countries, 89% (95% CI 81%, 93%); developed countries, 91% (95% CI 87%, 95%; p 0.43) and the region of the study (p 0.85). The diagnostic specificity of the typical CT findings for COVID-19 in studies of vaccinated patients was significantly lower than in studies of unvaccinated patients (68% vs. 91%, p 0.03). There was no significant difference in diagnostic performance according to the mean age, the proportion of male patients, study size, or CT classification system (p > 0.1, respectively). A sensitivity analysis confined to large studies showed similar pooled sensitivity and specificity (Additional file 1).

Table 2 Pooled estimates and meta-regression of diagnostic accuracy of typical CT findings
Full size table
Fig. 3
figure 3

Estimated predictive values of typical CT findings and rapid antigen test (RAT). The black lines indicate predictive values and 95% confidence intervals for RAT, and the blue lines indicate predictive values and 95% confidence intervals for typical CT findings. Square indicates summary points of sensitivity and specificity, and solid black curvilinear line passing through summary point is the estimated summary ROC curve. Black dashed line and blue dashed line represent 95% confidence region and prediction region respectively. *Proportion of prediction region to area under ROC curve = 35.8%

Full size image

A comparative analysis of diagnostic performance between typical CT findings and RAT is presented in Fig. 3. The PPV was significantly higher for RAT than for the typical CT findings, while there was no significant difference in NPV between RAT and the typical CT findings.

Sensitivity analysis for the diagnostic performance of the typical CT findings of COVID-19

The results of a sensitivity analysis for 28 studies, including more than 50 subjects among both COVID-19 cases and non-COVID-19 cases, each, were similar to those of the primary analysis [9, 20,21,22,23,24, 26,27,28, 30, 33,34,35,36,37,38, 41, 45,46,47,48, 50,51,52, 54,55,56,57] (Additional file 1: Fig. 4). The pooled sensitivity was 70% (95% CI 65%, 75%; I2 = 94%) and the pooled specificity was 91% (95% CI 88%, 94%; I2 = 95%) for the typical CT findings.

Interobserver agreement

Interobserver agreement for the typical CT findings was reported in 11 studies: seven studies reported diagnostic accuracy and the interobserver agreement together [9, 20, 22, 28, 37, 45, 51] and four studies solely reported the interobserver agreement [59, 60, 63, 64]. The reported κ values ranged from 0.52 to 0.93. The overall pooled estimate of κ values was 0.72 (95% CI 0.63, 0.81; I2 = 99%). However, the pooled estimate of κ values for the typical CT findings of the RSNA classification system (0.79; 95% CI 0.67, 0.91) was significantly higher than that of the CO-RADS system (0.64; 95% CI 0.55, 0.73) (p 0.05).

For the overall CT categories, the interobserver agreement was reported in 18 studies: 12 studies reported diagnostic accuracy and the interobserver agreement together [9, 20, 22, 25, 28, 36,37,38,39, 45, 51, 54] and six studies solely reported the interobserver agreement [59, 61,62,63,64,65]. They reported κ values ranged from 0.43 to 0.90. The pooled estimate of κ values was 0.67 (95% CI 0.61, 0.74; I2 = 99%). However, the pooled estimate of κ values for the overall CT findings of the RSNA classification system (0.74; 95% CI 0.65, 0.83) was significantly higher than that of the CO-RADS system (0.59; 95% CI 0.51, 0.67; P = 0.02) (Additional file 1: Fig. 5).

Publication bias

There was no obvious publication bias in the studies reporting diagnostic accuracy of the typical CT findings of COVID-19 (Additional file 1: Fig. 6A). For interobserver agreement, there was a tendency for a lower degree of agreement to be shown as the precision of the κ value in the binomial classification decreased. There was no significant asymmetry in the overall CT classifications (Additional file 1: Fig. 6B).

Discussion

This meta-analysis, which included 42 diagnostic performance studies from 18 developing and 24 developed countries, demonstrated that the pooled sensitivity was 70% (95% CI 65%, 74%) and the pooled specificity was 90% (95% CI 86%, 93%) for typical CT findings of COVID-19. In the meta-analysis of the RSNA classification system and CO-RADS in January 2021, the corresponding pooled estimates were 65% and 94% in the RSNA system (typical appearance; four studies from three countries) and 70% and 93% in CO-RADS (grade 5; six studies from three countries). We included a larger number of studies and merged the results of the typical findings in both systems. The diagnostic measure estimates of each system in the early observations were maintained in our results, while the 95% CIs became narrower. The pooled diagnostic performance did not differ by national income and region. Furthermore, the inter-reader agreement for dichotomizing CT findings into typical or not was substantial and higher than the categorical interpretation in both systems. These findings highlight that once typical findings of COVID-19 are defined in standardized reporting systems, radiologists can reproduce CT performance globally, despite geographic and resource variation.

In the early pandemic, the Fleischner Society advised using chest imaging for triaging suspects at risk of having moderate to severe COVID-19 when a point-of-care test was unavailable and resources were constrained [2]. However, whether chest CT could provide comparable accuracy for COVID-19 to point-of-care testing, as represented by RAT, remains underexplored. We compared the PPV and NPV of the typical CT findings and RAT based on pooled estimates for various disease prevalence rates: the NPV was almost identical between CT and RAT, and typical CT findings provided a lower PPV than RAT by up to 30–40%. When recollecting data from the massive Chinese surge of COVID-19 in early 2020, RAT was unavailable and PCR suffered from supply shortages and false-negative results, similar to shortages of large-scale laboratory testing capacities in low- and middle-income countries [66, 67]. The Chinese national guideline (trial version 5) temporarily used typical CT findings of viral pneumonia for making a clinical diagnosis of suspected COVID-19 cases in Wuhan and Hubei [68]. Given the pooled diagnostic CT performance, we may imagine why the first-line diagnosis/triage use of CT could be inevitably considered a practical supplemental option, although CT delivered radiation exposure [69] and it was unknown how well CT interpretation for typical COVID-19 findings was standardized in Wuhan and Hubei.

Chest CT findings can substantially overlap between COVID-19, influenza, and organizing pneumonia [70], and our findings on the pooled CT performance should be cautiously interpreted. Radiologists’ diagnostic CT accuracy for COVID-19 is low (~ 70%) if the three diseases have similar prevalence [70]. Fortunately, the incidence of influenza was historically low when COVID-19 predominated [71], and organizing pneumonia is an uncommon disease. Furthermore, vaccination [72] and the Omicron variant [73,74,75] can decrease diagnostic performance based on typical CT findings. Accordingly, the current pooled estimates are applicable to the early phase of the COVID-19 pandemic when vaccination was not sufficiently available and before the Omicron variant occurred.

The diagnostic accuracy of the typical findings among CT classification systems was almost identical between the RSNA and CO-RADS systems (p 0.72), in concordance with previous studies [10, 46, 59, 62]. In addition, higher CT categories showed higher diagnostic accuracy, similar to previous studies [10]. While the RSNA and CO-RADS systems have slightly different definitions of the typical category (such as the presence of subpleural sparing or thickened vessels), our study has demonstrated no significant difference in diagnostic accuracy between the two systems. Therefore, it is reasonable to consider the combined diagnostic ability of both systems in identifying typical COVID-19 pneumonia using a standardized classification approach. Meanwhile, the interobserver agreement was higher in the RSNA classification system than in the CO-RADS system (κ values, 0.79 vs 0.64). A previous study [46] surveyed preferences for the CT classification system and found that most participants preferred the RSNA system over the CO-RADS system. This is presumably because the RSNA classification system is more straightforward and user-friendly.

Our study had limitations. First, studies from low-income countries fundamentally lacking CT resources were not included in this study. Since few related studies are being conducted in low-income countries, further verification will be required based on the results of this study. Second, most of the studies were retrospectively designed and had a low vaccination rate, making it difficult to estimate the impact of vaccination on the diagnostic performance of the typical CT findings. Third, most studies did not provide information on the symptom presence, disease severity, and symptom onset of the included patients, although those factors could affect the diagnostic performance of CT findings.

In conclusion, the typical chest CT findings of COVID-19 based on standardized CT classification showed moderate sensitivity and high specificity globally, regardless of region and national income, and presented substantial interobserver agreement. If another pandemic occurs, radiology societies should prioritize providing standardized image interpretation for the pandemic disease’s typical findings as soon as possible. A standardized interpretation will play a crucial role in prompt diagnoses and triage until reference or point-of-care testing is sufficiently established.

Availability of data and materials

Archived datasets that are publicly available were not included in our study.

Abbreviations

CI:

Confidence interval

CO-RADS:

COVID-19 reporting and data system

QUADAS-2:

Quality Assessment of Diagnostic Accuracy Studies 2

RAT:

Rapid antigen test

ROC:

Receiver operating characteristic

RSNA:

Radiological Society of North America

RT-PCR:

Reverse transcription polymerase chain reaction

References

  1. Wang C, Horby PW, Hayden FG, Gao GF (2020) A novel coronavirus outbreak of global health concern. Lancet 395(10223):470–473. https://doi.org/10.1016/s0140-6736(20)30185-9

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Rubin GD, Ryerson CJ, Haramati LB et al (2020) The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner society. Radiology 296(1):172–180. https://doi.org/10.1148/radiol.2020201365

    Article  PubMed  Google Scholar 

  3. Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC (2020) Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019 (COVID-19): a review. JAMA 324(8):782–793. https://doi.org/10.1001/jama.2020.12839

    Article  CAS  PubMed  Google Scholar 

  4. Ghasemi S, Harmooshi NN, Rahim F (2022) Diagnostic utility of antigen detection rapid diagnostic tests for Covid-19: a systematic review and meta-analysis. Diagn Pathol 17(1):36. https://doi.org/10.1186/s13000-022-01215-6

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Yamayoshi S, Sakai-Tagawa Y, Koga M et al (2020) Comparison of rapid antigen tests for COVID-19. Viruses. https://doi.org/10.3390/v12121420

    Article  PubMed  PubMed Central  Google Scholar 

  6. Fenollar F, Bouam A, Ballouche M et al (2021) Evaluation of the panbio COVID-19 rapid antigen detection test device for the screening of patients with COVID-19. J Clin Microbiol. https://doi.org/10.1128/jcm.02589-20

    Article  PubMed  PubMed Central  Google Scholar 

  7. Kim H, Hong H, Yoon SH (2020) Diagnostic performance of CT and reverse transcriptase polymerase chain reaction for coronavirus disease 2019: a meta-analysis. Radiology 296(3):E145–E155

    Article  PubMed  Google Scholar 

  8. Simpson S, Kay FU, Abbara S et al (2020) Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19 endorsed by the society of thoracic radiology, the American College of Radiology, and RSNA-Secondary Publication. J Thorac Imaging 35(4):219–227. https://doi.org/10.1097/rti.0000000000000524

    Article  PubMed  Google Scholar 

  9. Prokop M, van Everdingen W, van Rees VT et al (2020) CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19-definition and evaluation. Radiology 296(2):E97-e104. https://doi.org/10.1148/radiol.2020201473

    Article  PubMed  Google Scholar 

  10. Kwee RM, Adams HJA, Kwee TC (2021) Diagnostic performance of CO-RADS and the RSNA classification system in evaluating COVID-19 at chest CT: a meta-analysis. Radiol Cardiothorac Imaging 3(1):e200510–e200510. https://doi.org/10.1148/ryct.2021200510

    Article  PubMed  PubMed Central  Google Scholar 

  11. Akl EA, Blažić I, Yaacoub S et al (2021) Use of chest imaging in the diagnosis and management of COVID-19: a WHO rapid advice guide. Radiology 298(2):E63–E69. https://doi.org/10.1148/radiol.2020203173

    Article  PubMed  Google Scholar 

  12. Liberati A, Altman DG, Tetzlaff J et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 151(4):W65-94. https://doi.org/10.7326/0003-4819-151-4-200908180-00136

    Article  PubMed  Google Scholar 

  13. World Bank Country and Lending Groups (2022). The World Bank. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups

  14. Coronavirus (COVID-19) Vaccinations (2022) Our world in data

  15. Whiting PF, Rutjes AW, Westwood ME et al (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155(8):529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009

    Article  PubMed  Google Scholar 

  16. Chu H, Cole SR (2006) Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol 59(12):1331–1332; author reply 1332–1333. https://doi.org/10.1016/j.jclinepi.2006.06.011

  17. Arshadi M, Fardsanei F, Deihim B et al (2022) Diagnostic accuracy of rapid antigen tests for COVID-19 detection: a systematic review with meta-analysis. Front Med (Lausanne) 9:870738. https://doi.org/10.3389/fmed.2022.870738

    Article  PubMed  Google Scholar 

  18. Abdolrahimzadeh Fard H, Mahmudi-Azer S, Sefidbakht S et al (2021) Evaluation of chest CT scan as a screening and diagnostic tool in trauma patients with coronavirus disease 2019 (COVID-19): a cross-sectional study. Emerg Med Int. https://doi.org/10.1155/2021/4188178

    Article  PubMed  PubMed Central  Google Scholar 

  19. Barbosa P, Bitencourt AGV, de Miranda GD, Almeida MFA, Chojniak R (2020) Chest CT accuracy in the diagnosis of SARS-CoV-2 infection: initial experience in a cancer center. Radiol Bras 53(4):211–215. https://doi.org/10.1590/0100-3984.2020.0040

    Article  PubMed  PubMed Central  Google Scholar 

  20. Bellini D, Panvini N, Rengo M et al (2021) Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study. Eur Radiol 31(4):1932–1940

    Article  CAS  PubMed  Google Scholar 

  21. Berkel BV, Vandevenne J, Coursier K et al (2021) Chest CT diagnosis of COVID-19: accuracy using CO-RADS and CT-involvement scoring. J Belgian Soc Radiol. https://doi.org/10.5334/JBSR.2342

    Article  Google Scholar 

  22. Borges Da Silva Teles G, Kaiser Ururahy Nunes Fonseca E, Yokoo P et al (2021) Performance of chest computed tomography in differentiating coronavirus disease 2019 from other viral infections using a standardized classification. J Thoracic Imaging 36(1):31–36. https://doi.org/10.1097/RTI.0000000000000563

    Article  PubMed  Google Scholar 

  23. Cengel F, Gurkan O, Calik M et al (2021) Diagnosis of the coronavirus disease 2019 with chest computed tomography: a retrospective inter-observer agreement study between radiologists and clinicians. Hong Kong J Emerg Med 28(1):15–21. https://doi.org/10.1177/1024907920968648

    Article  Google Scholar 

  24. Ciccarese F, Coppola F, Spinelli D et al (2020) Diagnostic accuracy of North America expert consensus statement on reporting CT findings in patients suspected of having COVID-19 infection: an Italian single-center experience. Radiol Cardiothorac Imaging 2(4):e200312

    Article  PubMed  PubMed Central  Google Scholar 

  25. de Jaegere TMH, Krdzalic J, Fasen B, Kwee RM, group C-CIS-ENs (2020) Radiological Society of North America chest CT classification system for reporting COVID-19 pneumonia: interobserver variability and correlation with reverse-transcription polymerase chain reaction. Radiol Cardiothorac Imaging 2(3):e200213

    Article  PubMed  PubMed Central  Google Scholar 

  26. De Smet K, De Smet D, Ryckaert T et al (2021) Diagnostic performance of chest CT for SARS-CoV-2 infection in individuals with or without COVID-19 symptoms. Radiology 298(1):E30–E37

    Article  PubMed  Google Scholar 

  27. Falaschi Z, Danna PSC, Arioli R et al (2020) Chest CT accuracy in diagnosing COVID-19 during the peak of the Italian epidemic: a retrospective correlation with RT-PCR testing and analysis of discordant cases. Eur J Radiol 130:109192. https://doi.org/10.1016/j.ejrad.2020.109192

    Article  PubMed  PubMed Central  Google Scholar 

  28. Fujioka T, Takahashi M, Mori M et al (2020) Evaluation of the usefulness of CO-RADS for chest CT in patients suspected of having COVID-19. Diagnostics 10(9):19

    Article  Google Scholar 

  29. Gross A, Heine G, Schwarz M, Thiemig D, Gläser S, Albrecht T (2021) Structured reporting of chest CT provides high sensitivity and specificity for early diagnosis of COVID-19 in a clinical routine setting. Br J Radiol. https://doi.org/10.1259/bjr.20200574

    Article  PubMed  Google Scholar 

  30. Guimaraes JGB, de Oliveira Salvador GL, Papp CL et al (2022) Diagnostic accuracy of CO-RADS in patients with suspected coronavirus disease-2019: a single center experience. Clin Imaging 86:7–12. https://doi.org/10.1016/j.clinimag.2022.02.005

    Article  PubMed  PubMed Central  Google Scholar 

  31. Gümüs T, Kabaoglu ZU, Coskun B, Kartal F, Artukoglu F, Atasoy KC (2021) Preoperative computerized tomography screening for COVID-19 pneumonia in asymptomatic patients: experiences from two centers. Jpn J Radiol 39(3):240–245. https://doi.org/10.1007/s11604-020-01061-w

    Article  CAS  PubMed  Google Scholar 

  32. Guner NG, Durmus E, Guneysu F, Aslan N, Yurumez Y (2021) The match of CO-RADS score and vaccination status of COVID-19 patients applying to the emergency room. Anaesthesia Pain Intensive Care 25(6):771–776. https://doi.org/10.35975/apic.v25i6.1701

    Article  CAS  Google Scholar 

  33. Hammer MM (2022) Real-world diagnostic performance of RSNA consensus reporting guidelines for findings related to COVID-19 on chest CT. AJR Am J Roentgenol 218(1):75–76. https://doi.org/10.2214/AJR.21.26560

    Article  PubMed  Google Scholar 

  34. Hermans JJR, Groen J, Zwets E et al (2020) Chest CT for triage during COVID-19 on the emergency department: myth or truth? Emerg Radiol 27(6):641–651. https://doi.org/10.1007/s10140-020-01821-1

    Article  PubMed  PubMed Central  Google Scholar 

  35. Inanc IH, Bursa N, Gultepe A, Bayramoglu M, Sabanoglu C, Inanc FA (2022) Association among CO-RADS score, co-morbid diseases, and short-term prognosis in COVID-19 infection. Eur Rev Med Pharmacol Sci 26(2):653–663. https://doi.org/10.26355/eurrev_202201_27892

    Article  CAS  PubMed  Google Scholar 

  36. Inui S, Kurokawa R, Nakai Y et al (2020) Comparison of chest CT grading systems in coronavirus disease 2019 (COVID-19) pneumonia. Radiol Cardiothorac Imaging 2(6):e200492

    Article  PubMed  PubMed Central  Google Scholar 

  37. Kavak S, Duymus R (2021) RSNA and BSTI grading systems of COVID-19 pneumonia: comparison of the diagnostic performance and interobserver agreement. BMC Med Imaging 21(1):143. https://doi.org/10.1186/s12880-021-00668-3

    Article  PubMed  PubMed Central  Google Scholar 

  38. Koşar Tunç M, Kiş N, İnce O, Kurtul Yildiz H, Önder H (2021) Diagnostic performance of COVID-19 reporting and data system classification across residents and radiologists: a retrospective study. J Comput Assist Tomogr 45(5):782–787. https://doi.org/10.1097/RCT.0000000000001172

    Article  PubMed  Google Scholar 

  39. Kurokawa R, Inui S, Gonoi W et al (2021) Standardized reporting systems of chest computed tomography in a population with low coronavirus disease 2019 prevalence: a retrospective comparative study. Heliyon 7(8):e07743

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Lang M, Som A, Mendoza DP et al (2020) Detection of unsuspected coronavirus disease 2019 Cases by computed tomography and retrospective implementation of the radiological society of North America/Society of Thoracic Radiology/American College of Radiology Consensus Guidelines. J Thorac Imaging 35(6):346–353

    PubMed  Google Scholar 

  41. Lieveld AWE, Azijli K, Teunissen BP et al (2021) Chest CT in COVID-19 at the ED: validation of the COVID-19 reporting and data system (CO-RADS) and CT severity score: a prospective, multicenter. Observ Study Chest 159(3):1126–1135. https://doi.org/10.1016/j.chest.2020.11.026

    Article  CAS  Google Scholar 

  42. Majeed T, Ali RS, Solomon J et al (2020) The role of the computed tomography (CT) thorax in the diagnosis of COVID-19 for patients presenting with acute surgical emergencies a single institute experience. Indian J Surg 82(6):1005–1010

    Article  PubMed  Google Scholar 

  43. Martinez-Fierro ML, Gonzalez-Fuentes C, Cid-Guerrero D et al (2022) Radiological findings increased the successful of COVID-19 diagnosis in hospitalized patients suspected of respiratory viral infection but with a negative first SARS-CoV-2 RT-PCR result. Diagnostics 12(3):11

    Article  Google Scholar 

  44. Miranda Magalhães Santos JM, Paula Alves Fonseca A, Pinheiro Zarattini Anastacio E, et al. (2020) Initial results of the use of a standardized diagnostic criteria for chest computed tomography findings in coronavirus disease 2019. J Comput Assist Tomogr 44(5):647–651. https://doi.org/10.1097/RCT.0000000000001054

    Article  PubMed  Google Scholar 

  45. Nair AV, McInnes M, Jacob B et al (2021) Diagnostic accuracy and inter-observer agreement with the CO-RADS lexicon for CT chest reporting in COVID-19. Emerg Radiol 28(6):1045–1054. https://doi.org/10.1007/s10140-021-01967-6

    Article  PubMed  PubMed Central  Google Scholar 

  46. O’Neill SB, Byrne D, Müller NL et al (2021) Radiological Society of North America (RSNA) expert consensus statement related to chest CT findings in COVID-19 versus CO-RADS: comparison of reporting system performance among chest radiologists and end-user preference. Can Assoc Radiol J 72(4):806–813. https://doi.org/10.1177/0846537120968919

    Article  PubMed  Google Scholar 

  47. Özel M, Aslan A, Araç S (2021) Use of the COVID-19 reporting and data system (CO-RADS) classification and chest computed tomography involvement score (CT-IS) in COVID-19 pneumonia. Radiol Med 126(5):679–687. https://doi.org/10.1007/s11547-021-01335-x

    Article  PubMed  PubMed Central  Google Scholar 

  48. Özer H, Kılınçer A, Uysal E et al (2021) Diagnostic performance of Radiological Society of North America structured reporting language for chest computed tomography findings in patients with COVID-19. Jpn J Radiol 39(9):877–888. https://doi.org/10.1007/s11604-021-01128-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Palmisano A, Scotti GM, Ippolito D et al (2021) Chest CT in the emergency department for suspected COVID-19 pneumonia. Radiol Med 126(3):498–502. https://doi.org/10.1007/s11547-020-01302-y

    Article  PubMed  Google Scholar 

  50. Palwa AR, Atiq-Ur-Rehman S, Nayyar B et al (2021) Chest CT severity score as an auxiliary grading tool to COVID-19 pneumonia imaging classification: a tertiary care experience in Pakistan. J Coll Physicians Surg Pak 31(1):14–20. https://doi.org/10.29271/jcpsp.2021.01.14

    Article  PubMed  Google Scholar 

  51. Rocha CO, Prioste TAD, Faccin CS et al (2022) Diagnostic performance of the RSNA-proposed classification for COVID-19 pneumonia versus pre-pandemic controls. Braz J Infect Dis. https://doi.org/10.1016/j.bjid.2021.101665

    Article  PubMed  Google Scholar 

  52. Schalekamp S, Bleeker-Rovers CP, Beenen LFM et al (2021) Chest CT in the emergency department for diagnosis of COVID-19 pneumonia: Dutch experience. Radiology 298(2):E98-e106. https://doi.org/10.1148/radiol.2020203465

    Article  PubMed  Google Scholar 

  53. Shirota G, Sato Y, Itoh D et al (2021) Pitfalls in chest CT findings of COVID-19 patients infected during hospitalisation. Clin Imaging 78:146–153. https://doi.org/10.1016/j.clinimag.2021.03.017

    Article  PubMed  PubMed Central  Google Scholar 

  54. Silva CF, Alegria J, Ramos C et al (2021) RSNA-STR-ACR consensus statement for COVID-19 CT patterns: interreader agreement in 240 consecutive patients and association with RT-PCR status. J Comput Assist Tomogr 45(3):485–489. https://doi.org/10.1097/RCT.0000000000001162

    Article  PubMed  Google Scholar 

  55. Turcato G, Zaboli A, Panebianco L et al (2021) Clinical application of the COVID-19 reporting and data system (CO-RADS) in patients with suspected SARS-CoV-2 infection: observational study in an emergency department. Clin Radiol 76(1):74.e23-74.e29. https://doi.org/10.1016/j.crad.2020.10.007

    Article  CAS  PubMed  Google Scholar 

  56. Valentin B, Steuwe A, Wienemann T et al (2022) Applicability of CO-RADS in an anonymized cohort including early and advanced stages of COVID-19 in comparison to the recommendations of the German radiological society and radiological society of North America. Rofo. https://doi.org/10.1055/a-1740-4310

    Article  PubMed  Google Scholar 

  57. Vicini S, Panvini N, Bellini D et al (2021) Radiographers and COVID-19 pneumonia: diagnostic performance using CO-RADS. Radiography 27(4):1078–1084. https://doi.org/10.1016/j.radi.2021.04.010

    Article  CAS  PubMed  Google Scholar 

  58. Yassin A, Abdelkader MA, Mohammed RM, Nagi MAMA (2022) CT Patterns among covid-19 patients during the second wave: a single institute study. Egypt J Hosp Med 86(1):165–171. https://doi.org/10.21608/EJHM.2022.210793

    Article  Google Scholar 

  59. Abdel-Tawab M, Basha MAA, Mohamed IAI et al (2021) Comparison of the CO-RADS and the RSNA chest CT classification system concerning sensitivity and reliability for the diagnosis of COVID-19 pneumonia. Insights Imaging 12(1):55. https://doi.org/10.1186/s13244-021-00998-4

    Article  PubMed  PubMed Central  Google Scholar 

  60. Byrne D, Neill SBO, Müller NL et al (2021) RSNA expert consensus statement on reporting chest CT findings related to COVID-19: interobserver agreement between chest radiologists. Can Assoc Radiol J 72(1):159–166. https://doi.org/10.1177/0846537120938328

    Article  PubMed  Google Scholar 

  61. Dilek O, Kaya O, Akkaya H et al (2021) Diagnostic performance and interobserver agreement of CO-RADS: evaluation of classification in radiology practice. Diagn Interv Radiol 27(5):615–620. https://doi.org/10.5152/dir.2021.201032

    Article  PubMed  PubMed Central  Google Scholar 

  62. Fonseca EKUN, Loureiro BMC, Strabelli DG et al (2021) Evaluation of the RSNA and CORADS classifications for COVID-19 on chest computed tomography in the Brazilian population. Clinics (Sao Paulo) 76:e2476. https://doi.org/10.6061/clinics/2021/e2476

    Article  PubMed  Google Scholar 

  63. Sushentsev N, Bura V, Kotnik M et al (2020) A head-to-head comparison of the intra- and interobserver agreement of COVID-RADS and CO-RADS grading systems in a population with high estimated prevalence of COVID-19. BJR Open 2(1):20200053

    PubMed  PubMed Central  Google Scholar 

  64. Hadied MO, Patel PY, Cormier P et al (2020) Interobserver and intraobserver variability in the CT assessment of COVID-19 based on RSNA consensus classification categories. Acad Radiol 27(11):1499–1506. https://doi.org/10.1016/j.acra.2020.08.038

    Article  PubMed  PubMed Central  Google Scholar 

  65. Atta H, Hasan HA, Elmorshedy R, Gabr A, Abbas WA, El-Barody MM (2021) Validation of imaging reporting and data system of coronavirus disease 2019 lexicons CO-RADS and COVID-RADS with radiologists’ preference: a multicentric study. Egypt J Radiol Nucl Med 52(1):109. https://doi.org/10.1186/s43055-021-00485-2

    Article  Google Scholar 

  66. Mohanan M, Malani A, Krishnan K, Acharya A (2021) Prevalence of COVID-19 In rural versus urban areas in a low-income country: findings from a state-wide study in Karnataka India. medRxiv. https://doi.org/10.1101/2020.11.02.20224782

    Article  Google Scholar 

  67. Yusuf L, Appeaning M, Amole TG et al (2021) Rapid, cheap, and effective COVID-19 diagnostics for Africa. Diagnostics 11(11):2105

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Zu ZY, Jiang MD, Xu PP et al (2020) Coronavirus disease 2019 (COVID-19): a perspective from China. Radiology 296(2):E15–E25. https://doi.org/10.1148/radiol.2020200490

    Article  PubMed  Google Scholar 

  69. Lee JH, Hong H, Kim H, Lee CH, Goo JM, Yoon SH (2021) CT examinations for COVID-19: a systematic review of protocols, radiation dose, and numbers needed to diagnose and predict. J Korean Soc Radiol 82(6):1505–1523

    Article  Google Scholar 

  70. Garrana SH, Som A, Ndakwah GS et al (2021) Comparison of chest CT findings of COVID-19, influenza, and organizing pneumonia: a multireader study. AJR Am J Roentgenol 217(5):1093–1102. https://doi.org/10.2214/AJR.21.25640

    Article  PubMed  Google Scholar 

  71. Olsen SJ, Winn AK, Budd AP et al (2021) Changes in influenza and other respiratory virus activity during the COVID-19 pandemic-United States 2020–2021. MMWR Morb Mortal Wkly Rep 70(29):1013–1019. https://doi.org/10.15585/mmwr.mm7029a1

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Polyakov NJ, Som A, Mercaldo ND, Di Capua J, Little BP, Flores EJ (2022) True-positive rate of RSNA typical chest CT findings for COVID-19 pneumonia in an increasingly vaccinated population. Radiology. https://doi.org/10.1148/radiol.220680

    Article  PubMed  Google Scholar 

  73. Tsakok MT, Watson RA, Saujani SJ et al (2022) Chest CT and hospital outcomes in patients with omicron compared with delta variant SARS-CoV-2 infection. Radiology. https://doi.org/10.1148/radiol.220533

    Article  PubMed  Google Scholar 

  74. Yoon SH, Lee JH, Kim BN (2022) Chest CT findings in hospitalized patients with SARS-CoV-2: delta versus omicron variants. Radiology. https://doi.org/10.1148/radiol.220676

    Article  PubMed  PubMed Central  Google Scholar 

  75. Lee JE, Hwang M, Kim YH et al (2022) SARS-CoV-2 variants infection in relationship to imaging-based pneumonia and clinical outcomes. Radiology. https://doi.org/10.1148/radiol.221795

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge Andrew Dombrowski, PhD (Compecs, Inc.) for his assistance in improving the use of English in this manuscript.

Funding

This work was supported by the Soonchunhyang University Research Fund and the Korean Medical Device Development Fund grant funded by the Korea government (the Ministry of Science and ICT, the Ministry of Trade Industry and Energy, the Ministry of Health & Welfare, Republic of Korea, the Ministry of Food and Drug Safety) (Project Number: 202011A03).

Author information

Author notes

  1. Bo Da Nam and Hyunsook Hong contributed equally to this work as first authors.

Authors and Affiliations

  1. Department of Radiology, Soonchunhyang University College of Medicine, Soonchunhyang University Seoul Hospital, Seoul, Republic of Korea

    Bo Da Nam

  2. Medical Research Collaborating Center, Seoul National University Hospital, Seoul, Republic of Korea

    Hyunsook Hong

  3. Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, Republic of Korea

    Soon Ho Yoon

Authors
  1. Bo Da Nam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyunsook Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soon Ho Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: SHY. Data curation: BDN, SHY. Formal analysis: all authors. Funding acquisition: BDN, SHY. Investigation: all authors. Methodology: HH. Project administration: SHY. Resources: SHY, BDN. Software: HH. Supervision: SHY. Validation: all authors. Visualization: all authors. Writing-original draft: all authors. Writing-review & editing: all authors. All the authors have approved the manuscript and agree with submission to the journal. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Soon Ho Yoon.

Ethics declarations

Ethics approval and consent to participate

Not applicable. Since this study was a systematic review and meta-analysis, informed consent was not required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests related to the present study. Not related to the present article: SHY is the chief medical officer for Medical IP and holds a stock option in the firm.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Supplemental Material.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nam, B.D., Hong, H. & Yoon, S.H. Diagnostic performance of standardized typical CT findings for COVID-19: a systematic review and meta-analysis. Insights Imaging 14, 96 (2023). https://doi.org/10.1186/s13244-023-01429-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13244-023-01429-2

Keywords