CanadiEM Journal Club E04 Systematic reviews and meta analyses show notes

Welcome back to Journal Club by CanadiEM! In this episode we go over an approach to systematic reviews and meta analyses based on Oxford centre of EBM, and learn about diagnosing pneumothorax with ultrasound vs X-ray

Using the Oxford centre of EBM tool, we will ask:

1. What question(s) did the systematic review address?
2. Is it likely that important, relevant studies were missed?
3. Were the criteria used to select articles for inclusion appropriate?
4. Were the included studies sufficiently valid for the type of question asked?
5. Were the results similar from study to study?
6. What were the results?
7. What is the clinical significance of the results?

and then a clinical pearl on pneumothorax!!

Hosts: 

• Dakoda Herman
• Jayneel Limbachia
• Jake Domm

Paper: “Chest ultrasonography versus supine chest radiography for diagnosis of pneumothorax in trauma patients in the emergency department” Cochrane Database of Systematic Reviews by Chan KK, Joo DA, McRae AD, Takwoingi Y, Premji ZA, Lang E, Wakai A

What question(s) did the systematic review address?

P: Trauma patients in the ER

I: chest ultrasonography by non rad physicians

C: Chest xray

O: diagnosis of pneumothorax, improved patient safety 

• Secondary: investigate potential sources of hetero such as type of CUS operator, type of trauma, type of US probe on test accuracy

T: inception to 10 April 2020

Is it unlikely that important, relevant studies were missed?

• This study included prospective, paired comparative  accuracy studies in which patients were suspected of having pneumothorax. Patients must have undergone both CUS by a frontline non-radiologist and CXR, as well as CT of the chest or tube thoracostomy as the reference standard.
• The authors carried out systematic searches in the following electronic databases: Cochrane Database of Systematic Reviews; Cochrane Central Register of Controlled Trials; MEDLINE; Embase; Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus; Database of Abstracts of Reviews of EIects; Web of Science core collection (which includes: Science Citation Index Expanded; Social Sciences Citation Index; Arts & Humanities Citation Index; Conference Proceedings Citation Index - Science; Conference Proceedings Citation Index - Social Sciences & Humanities; and Emerging Sources Citation Index; and Clinicaltrials.gov from database creation to April 2020.
• The authors also handsearched reference lists of included articles and reviews, retrieved via electronic searching, for potentially eligible studies. Additionally, they carried out forward citation searching of relevant articles in Google Scholar and looked at the “Related articles” on PubMed.
• They did not limit the search to Englsih language only and included articles published in all languages.
• Their search strategy in volved the use of MeSH terms such as Pneumothorax, Radiography, Ultrasonography, and focused assessment with ultrasonography for trauma. They also used many text words.
• Using this search strategy 3473 records were identified. 1180 duplicated records were removed, leaving 2293 records to be screened. These records were screened by two of the authors for their relevance, when there was a discrepancy a third author decided whether to include the record or not. 2268 records were excluded, leaving 25 full-text articles that were assessed for eligibility. 12 studies were excluded - 5 missing CUS/CXR/CT chest/chest tube, 4 CUS not performed by frontline non-radiologist physicians, 2 wrong patient population, 1 wrong study design. A total of 13 studies were included in qualitative and quantitative analysis. 9 studies using patients as units of analysis included in primary analysis. 4 studies using lung fields as units of analysis included in secondary analysis.
• Authors provide a nice figure depicting this.
• Authors did not contact experts for unpublished data but were very thorough and transparent in their search strategy. I think it is unlikely that important, relevant studies were missed.

Were the criteria used to select articles for inclusion appropriate?

• The authors of this study clearly outlined their study inclusion and exclusion criteria.
• They included prospective, comparative accuracy studies in which patients were suspected of having pneumothorax. 
• They included trauma patients in the emergency department setting. Patients must have undergone both CUS by frontline non-radiologist physicians and CXR as index tests, as well as CT of the chest or tube thoracostomy as the reference standard. The two main index tests were CUS completed by a frontline nonradiologist physician and CXR, both being performed in the supine position. If data on specific CUS findings (such as the absence of lung sliding, absence of B-lines or comet-tail artefact, presence of lung point, and absence of lung pulse) were available, they planned to assess the diagnostic accuracy of these individual CUS findings. The target condition was traumatic pneumothorax of any severity. They defined a pneumothorax identified on CT scan of the chest or via clinical findings of a rush of air or bubbling in a chest drain after tube thoracostomy as the reference standard.
• The authors excluded studies involving participants with already diagnosed pneumothorax (i.e. case-control studies); studies involving participants with nontraumatic pneumothorax; studies involving participants who had already been treated with tube thoracostomy; and studies in which a frontline non-radiologist physician did not perform CUS.
• These criteria were appropriate for inclusion. However, many studies did differ in their units of measurement - lung fields vs. patients, which could introduce significant bias into the results of individual studies.

Were the included studies sufficiently valid for the type of question asked?

• The authors used the QUADAS-2 tool to assess risk of bias and the applicability of each included study. 
• This tool assesses risk of bias in four domains: patient selection; index tests; reference standard; and flow and timing. In addition, they examined concerns about applicability in the first three domains. They tailored the tool to their review question - one of the signalling questions in the patient selection domain was not applicable because they excluded case-control studies; therefore they deleted this question from the tool. 
• Two review authors performed the assessments independently. They discussed and resolved any disagreements that arose through consultation with a third review author.
• They included a figure describing their assessments of study quality.
• Of the nine studies that we included in the primary analysis:
• • One had a low risk of bias, two had an unclear risk, and six had high risk of bias in the patient selection domain, mostly due to convenience sampling or inappropriate exclusion criteria, such as excluding haemodynamically unstable patients, lack of access to CUS, chest wall injuries precluding CUS, or if CT was not indicated. 
  • The risk of bias in the interpretation of CUS results was low in five studies, unclear in two studies, and high in two studies; this was related to unclear blinding m