Seminars in Roentgenology
Volume 44, Issue 3 , Pages 209-213, July 2009

Evidence-based Practice Journal Club: How We Do It

  • Aine Marie Kelly, MD, MS, MRCP, FRCR
    • ,
  • Paul Cronin, MD, MS, MRCPI, FRCR

      Affiliations

    • Corresponding Author InformationAddress reprint requests to Paul Cronin, MD, MS, MRCPI, FRCR, Department of Radiology, Division of Cardiothoracic Radiology, University of Michigan Hospitals, B1 132G Taubman Center/5302, 1500 East Medical Center Drive, Ann Arbor, MI 48109-5302

Department of Radiology, Division of Cardiothoracic Radiology, University of Michigan Medical Center, Ann Arbor, MI

Article Outline

 

Our evidence-based practice (EBP) journey started in 2006, when we were approached by the associate residency director, and asked to participate in a journal club. Inspired by our experience of gaining knowledge of the principles and application of evidence-based medicine (EBM), we decided to run an EBP journal club with the radiology residents at the University of Michigan. We were delighted to accept this new challenge. We had both attended a part-time Masters in Clinical Research Design and Statistical Analysis at the University of Michigan School of Public Health, which gave us sufficient methodological and statistical background and stimulated our appetite for evidence-based medicine and critical appraisal.1 We had also attended the course for teachers in evidence-based medicine at the Centre for Evidence-Based Medicine in Oxford, UK.2 The Centre for Evidence-based Medicine in Oxford workshop is not intended to serve as an introduction to evidence-based medicine itself but is aimed at physicians and other health care professionals, who already have some knowledge of critical appraisal and experience in the practice of evidence-based health care and who want to explore issues around teaching evidence-based medicine. This workshop is aimed at those who are committed to patient-centered learning, believe in the process of small-group learning, and take an active role in small group learning. Armed with this knowledge, we set about applying these methods to the journal club. In addition the Accreditation Council for Graduate Medical Education (ACGME) has identified 6 core competencies in the curriculum for resident development (patient care, medical knowledge, interpretation and communication, professionalism, practice-based learning and improvement, and systems-based practice).3 Knowledge of the principles and application of EBM is explicitly included under the competencies of “medical knowledge” and “practice-based learning and improvement.” The Evidence-based Medicine Journal Club has been a great success and is now in its second year, and we would encourage anybody interested in doing this to start an EBP journal club at their own institution.

Back to Article Outline

The University of Michigan Evidence-based Medicine/Evidence-based Radiology Journal Club 

The evidence-based medicine/evidence based radiology (EBR) journal club at the University of Michigan series runs over 6 sessions, approximately every second month (in a 12-month period). The need to teach other core curricular information limits the frequency of the EBP journal club. The journal club occupies 2 noon conference slots (90 minutes) and comprises short didactic component(s) (maximum 20 minutes), presentation of articles (1 or 2, time permitting) by the residents, and a critical appraisal session (with active audience participation), reviewing the articles covered, using the STAndards for the Reporting of Diagnostic accuracy studies (STARD) criteria (Table 1) for a diagnostic radiology article, which is most of the articles reviewed.4, 5, 6

Table 1. The STARD Checklist for the Reporting of Studies of Diagnostic Accuracy
Section and TopicItem No.Question to be Answered
Title/abstract/keywords1Identify the article as a study of diagnostic accuracy (recommend mesh heading “sensitivity and specificity”)
Introduction2State the research questions or study aims, such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups
Methods Describe
Participants3The study population: the inclusion and exclusion criteria, setting, and locations where the data were collected
4Participant recruitment: was recruitment based on presenting symptoms, results from previous tests, or the fact that the participants had received the index tests or the reference standard?
5Participant sampling: was the study population a consecutive series of participants defined by the selection criteria in items 3 and 4? If not, specify how participants were further selected
6Data collection: was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)?
Test methods7The reference standard and its rationale.
8Technical specifications of material and methods involved, including how and when measurements were taken, and/or cite references for index tests and reference standard
9Definition of and rationale for the units, cutoffs, and/or categories of the results of the index tests and the reference standard
10The number, training, and expertise of the persons executing and reading the index tests and the reference standard
11Whether or not the readers of the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers
Statistical methods12Methods for calculating or comparing measures of diagnostic accuracy, and the statistical methods used to quantify uncertainty (eg, 95% confidence intervals)
13Methods for calculating test reproducibility, if done
Results Report
Participants14When study was done, including beginning and ending dates of recruitment
15Clinical and demographic characteristics of the study population (eg, age, sex, spectrum of presenting symptoms, comorbidity, current treatments, recruitment centers).
16The number of participants satisfying the criteria for inclusion that did or did not undergo the index tests and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended)
Test results17Time interval from the index tests to the reference standard, and any treatment administered between
18Distribution of severity of disease (define criteria) in those with the target condition; other diagnoses in participants without the target condition
19A cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard
20Any adverse events from performing the index tests or the reference standard
Estimates21Estimates of diagnostic accuracy and measures of statistical uncertainty (eg, 95% confidence intervals)
22How indeterminate results, missing responses, and outliers of the index tests were handled
23Estimates of variability of diagnostic accuracy between subgroups of participants, readers, or centers, if done
24Estimates of test reproducibility, if done
Discussion25Discuss the clinical applicability of the study findings

The aims of the Evidence-based Medicine/Radiology Journal Club are as follows:

1.Formulate answerable questions

2.Search effectively for evidence

3.Critically appraise evidence about diagnosis, treatment, and etiology for its validity and applicability

4.Understand basic measures of efficacy, such as sensitivity, specificity, predictive values, likelihood ratios, and numbers needed to treat

5.Understand the basics of decision analysis, guidelines, meta-analysis, and cost-effectiveness analysis.

The residents choose the articles to present and volunteer to present (and those that do so have been extremely enthusiastic so far!). They are asked to base the article on a clinical problem they have encountered, or a pressing or difficult on-call issue. What is innovative about this journal club is getting the residents to choose the topic and articles themselves. They usually choose a clinical problem that they have encountered on-call as the topic, so there is buy-in from the residents in general and a great interest in the chosen topic as many may have encountered the same problem on-call. This is important because when the residents are on-call, there is often less faculty supervision, patients are often sicker, and important decisions need to be made quickly. When on-call, there is usually little time to search for and appraise the evidence in the literature. We meet with the residents in advance to ensure that the article is suitable to appraise and that it will generate discussion from the audience.

Although Evidence-based Medicine and Critical Appraisal are part of the core competencies in the curriculum for resident development according to accreditation council for graduate medical education (ACGME), until this journal club came about, there has not been a formalized series of lectures at the resident noon conference on EBM and Critical Appraisal.3 Therefore, offering a conceptual framework of evidence-based medicine and seeing evidence-based medicine in practice is an innovation in itself. In general, standard EBM is usually taught with regard to medicine (as its name suggests) where therapeutic trials or randomized controlled trials, etc, are the mainstay of the literature. In radiology, which is predominantly concerned with diagnostic and imaging studies, there are differences. Therefore, what is truly innovative about this journal club is that we have transferred the principles of EBM to create a curriculum specific to diagnostic test and imaging and developed an EBR journal club.

Evidence-based medicine is part of the process of lifelong, self-directed learning in which caring for patients leads to the search for critical appraisal and incorporation into practice of valid and clinically important information about diagnosis, prognosis, therapy, and other clinical and health-care issues. The teaching of the University of Michigan EBR journal club is through different educational models for teaching with small group sessions (in which participants interact and develop their EBM knowledge and skills) and short plenary sessions (to fill in background knowledge gaps and discuss core issues in EBM/EBR and statistical methodology). However, plenary sessions are a minority of the learning experience with most of the journal club spent in interactive discussion in small groups with Drs Cronin and Kelly acting as facilitators of the learning process rather than traditional teachers.

The journal club begins with a short (maximum 10 minutes) didactic presentation on the components on Evidence-based Medicine (and Radiology), such as follows:

1.Evidence-based medicine—an overview

2.Asking an answerable question and searching the literature (steps 1 and 2)

3.Using the PubMed and MEDLINE search engines

4.Critically appraising the retrieved literature on diagnostic tests (for Diagnostic Radiology)

5.Critically appraising the retrieved literature on therapy (for Interventional Radiology)

6.Critically appraising the retrieved literature on Meta-analysis

This is followed with a short (maximum 10 minutes) didactic presentation on Statistical Methodology, such as follows:
1.“A Review of Available Statistical Tests for Continuous Variables”

2.“A Review of Available Statistical Tests for Categorical Variables”

3.“A Review of Available Statistical Tests for Dichotomous Variables”

4.“Appraising the Validity of a Diagnostic Study: Sensitivity, Specificity, SpPin, SnNout, Positive Predictive Value, Negative Predictive Value and Accuracy”

5.“Appraising the validity of a diagnostic study: Positive Likelihood Ratio and Negative Likelihood Ratio, and Conditional Probability”

6.“The ROC curve and The Kappa statistic”

 These plenary sessions are given as we feel that a short didactic component is still necessary to provide a framework or outline of what we will be doing. Also, we feel that these sessions are helpful to aid in the critical appraisal process especially as some residents may not be familiar with the evidence-based medicine process and critical appraisal of the literature.

Prior to starting the journal club, we e-mailed all the residents with papers from the users' guide to the medical literature series in Journal of the American Medical Association,7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36 and the evidence-based practice in radiology series in Radiology to give the residents background knowledge of evidence-based practice and critical appraisal of the literature.37, 38, 39, 40, 41, 42

Next, a resident (or 2 if time permits) will present a very short PowerPoint presentation on the article to be discussed. The resident will have met with us in advance to plan the talk and to ensure we include the salient points. This process gives the resident the experience of creating a PowerPoint presentation and presenting it to a group of peers.

As mentioned above, all the residents receive the PDF of the article by e-mail 1 to 2 weeks in advance of the journal club along with the STARD criteria and relevant articles on Evidence-based Medicine/Radiology, to help the residents critically appraise the article.

We use small group discussion and interactive learning. Following this presentation, the article is critically appraised by the residents in an adult-centered learning environment. In this environment, the residents are learning material that they see as relevant to their future practice. Application to the real world and their day-to-day activities is necessary. The learning experience must provide support from peers and not have fear of judgment during the learning process. We sit in among the group to have an informal and nonintimidating environment. We act as facilitators to keep the discussion going at a reasonable pace and to ensure that we stay focused. The group and a few residents in particular get very animated. It has been very reassuring, as the residents seem eager to debate and discuss the issues that interest them and do not hold back! Timely and appropriate feedback is given. The residents participate in small group discussion from this point onward with each given the STARD questions for Title/Abstract/Keywords/Introduction, Methods, Results, Discussion, etc.4, 5, 6 The residents discuss the STARD criteria (and any questions that arise) among themselves in their respective groups before presenting them to the group at large (Table 1).

The teaching process is adult-centered learning. The learning activities move beyond understanding to application, analysis, synthesis, and application. Adults bring with them a wide range of previous experience, knowledge, competencies, and interests to the learning process, which must accommodate them. Transfer of learning is not automatic and must be facilitated. Follow-up support is provided to enable transfer of learning into daily practice with open discussion and debate encouraged. Adult learners are ideal students for problem-solving and case-based education, in which the learner must integrate all their skills and knowledge to reach the goal of solving the problem. Adult-centered learning is life-centered rather than subject-centered. It is an active process and learning is self-directed. Learners engage with their experience through action (active experimentation) or reflection (reflective observation). Therefore, we have tried to structure the journal club so that learning is self-directed, as adult learners tend to have a rich reservoir of experience that can serve as a resource for learning. Adult learns tend to be affected by the need to know or do something and tend to have a life-, task-, or problem-centered orientation to learning as opposed to a subject matter orientation and are generally motivated to learn from within (internally/intrinsically) as opposed to being obligated, or subject to, external or extrinsic forces. Again, we have tried to structure the journal club to reflect this.

Back to Article Outline

Conclusions 

In summary, we feel that starting and running an evidence-based medicine/Radiology and Appraising the Literature Journal Club is important because Evidence-based Medicine and Critical Appraisal are part of the core competencies in the curriculum for resident development according to ACGME. It is important for residents (and all health-care providers) to have some knowledge of the principles and application of EBM. The residents choose an article based on a clinical problem they have encountered, or a pressing or difficult issue, such as an on-call issue. This applies learning to practical applications with issue-centered curricula.

We use multiple and diverse sources of information and a variety of teaching formats. The teaching process integrates thinking and learning in a problem-centered rather than content-oriented manner. There is also a mutual respect and equality for learners. We incorporate and promote dialogue and openness during discussions and recognize the value of experience in contributing to learning. The journal club includes topics with active learning (as opposed to only didactic lectures and/or passive learning). There is also built-in a mechanism for feedback and evaluation.

We feel that this process has resulted in the residents having a greater understanding of the practice of EBM/EBR and critical appraisal of the literature but because this is done using the intrinsic motivation of the residents and as a shared responsibility for the learning objectives in a nonprescriptive manner that is open to change, they have built a process of value that allows the tools for lifelong learning for all who participate, including the facilitators. The Evidence-based Medicine Journal Club at the University of Michigan has been a great success and is now in its second year, and we would encourage anybody interested in doing this to start an EBP journal club at their own institution.

Back to Article Outline

References 

  1. Masters in clinical research design and statistical analysis at the University of Michigan School of Public Health. Accessed: February 20, 2009 http://www.sph.umich.edu/biostat/programs/clinical-stat/
  2. The Course for Teachers in Evidence Based Medicine at the Centre for Evidence Based Medicine in Oxford, UK. Accessed: February 20, 2009 http://www.cebm.net/index.aspx?o=1732
  3. The Accreditation Council for Graduate Medical Education (ACGME): Core competencies. Accessed: February 20, 2009 http://www.acgme.org/outcome/comp/GeneralCompetenciesStandards21307.pdf
  4. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. AJR Am J Roentgenol. 2003;181:51–55
  5. Bossuyt PM, Reitsma JB, Bruns DE, et al. Toward complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Acad Radiol. 2003;10:664–669
  6. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Radiology. 2003;226:24–28
  7. Guyatt GH, Rennie D. Users' guides to the medical literature. J Am Med Assoc. 1993;270:2096–2097
  8. Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature (I. How to get started. The Evidence-Based Medicine Working Group). J Am Med Assoc. 1993;270:2093–2095
  9. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature (II. How to use an article about therapy or prevention. A. Evidence-Based Medicine Working Group: Are the results of the study valid?). J Am Med Assoc. 1993;270:2598–2601
  10. Jaeschke R, Guyatt G, Sackett DL. Users' guides to the medical literature (III. How to use an article about a diagnostic test. A. Evidence-Based Medicine Working Group:Are the results of the study valid?). J Am Med Assoc. 1994;271:389–391
  11. Levine M, Walter S, Lee H, et al. Users' guides to the medical literature (IV. How to use an article about harm. Evidence-Based Medicine Working Group). J Am Med Assoc. 1994;271:1615–1619
  12. Laupacis A, Wells G, Richardson WS, et al. Users' guides to the medical literature (V. How to use an article about prognosis. Evidence-Based Medicine Working Group). J Am Med Assoc. 1994;272:234–237
  13. Oxman AD, Cook DJ, Guyatt GH. Users' guides to the medical literature (VI. How to use an overview. Evidence-Based Medicine Working Group). J Am Med Assoc. 1994;272:1367–1371
  14. Richardson WS, Detsky AS. Users' guides to the medical literature (VII. How to use a clinical decision analysis. A. Evidence-Based Medicine Working Group: Are the results of the study valid?). J Am Med Assoc. 1995;273:1292–1295
  15. Richardson WS, Detsky AS. Users' guides to the medical literature (VII. How to use a clinical decision analysis. B. Evidence-Based Medicine Working Group: What are the results and will they help me in caring for my patients?). J Am Med Assoc. 1995;273:1610–1613
  16. Hayward RS, Wilson MC, Tunis SR, et al. Users' guides to the medical literature (VIII. How to use clinical practice guidelines. A. The Evidence-Based Medicine Working Group: Are the recommendations valid?). J Am Med Assoc. 1995;274:570–574
  17. Wilson MC, Hayward RS, Tunis SR, et al. Users' guides to the medical literature (VIII. How to use clinical practice guidelines. B. The Evidence-Based Medicine Working Group: What are the recommendations and will they help you in caring for your patients?). J Am Med Assoc. 1995;274:1630–1632
  18. Guyatt GH, Sackett DL, Sinclair JC, et al. Users' guides to the medical literature (IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group). J Am Med Assoc. 1995;274:1800–1804
  19. Naylor CD, Guyatt GH. Users' guides to the medical literature (X. How to use an article reporting variations in the outcomes of health services. The Evidence-Based Medicine Working Group). J Am Med Assoc. 1996;275:554–558
  20. Naylor CD, Guyatt GH. Users' guides to the medical literature (XI. How to use an article about a clinical utilization review. Evidence-Based Medicine Working Group). J Am Med Assoc. 1996;275:1435–1439
  21. Guyatt GH, Naylor CD, Juniper E, et al. Users' guides to the medical literature (XII. How to use articles about health-related quality of life. Evidence-Based Medicine Working Group). J Am Med Assoc. 1997;277:1232–1237
  22. Drummond MF, Richardson WS, O'Brien BJ, et al. Users' guides to the medical literature (XIII. How to use an article on economic analysis of clinical practice. A. Evidence-Based Medicine Working Group: Are the results of the study valid?). J Am Med Assoc. 1997;277:1552–1557
  23. Dans AL, Dans LF, Guyatt GH, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XIV. How to decide on the applicability of clinical trial results to your patient). J Am Med Assoc. 1998;279:545–549
  24. Richardson WS, Wilson MC, Guyatt GH, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XV. How to use an article about disease probability for differential diagnosis). J Am Med Assoc. 1999;281:1214–1219
  25. Guyatt GH, Sinclair J, Cook DJ, et al. Evidence-Based Medicine Working Group Cochrane Applicability Methods Working Group Users' guides to the medical literature (XVI. How to use a treatment recommendation). J Am Med Assoc. 1999;281:1836–1843
  26. Barratt A, Irwig L, Glasziou P, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XVII. How to use guidelines and recommendations about screening). J Am Med Assoc. 1999;281:2029–2034
  27. Randolph AG, Haynes RB, Wyatt JC, et al. Users' guides to the medical literature (XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system). J Am Med Assoc. 1999;282:67–74
  28. Bucher HC, Guyatt GH, Cook DJ, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XIX. Applying clinical trial results; A. How to use an article measuring the effect of an intervention on surrogate end points). J Am Med Assoc. 1999;282:771–778
  29. McAlister FA, Laupacis A, Wells GA, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XIX. Applying clinical trial results. B. Guidelines for determining whether a drug is exerting (more than) a class effect). J Am Med Assoc. 1999;282:1371–1377
  30. McAlister FA, Straus SE, Guyatt GH, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XX. Integrating research evidence with the care of the individual patient). J Am Med Assoc. 2000;283:2829–2836
  31. Hunt DL, Jaeschke R, McKibbon KA Evidence-Based Medicine Working Group. Users' guides to the medical literature (XXI. Using electronic health information resources in evidence-based practice). J Am Med Assoc. 2000;283:1875–1879
  32. McGinn TG, Guyatt GH, Wyer PC, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XXII: How to use articles about clinical decision rules). J Am Med Assoc. 2000;284:79–84
  33. Giacomini MK, Cook DJ Evidence-Based Medicine Working Group. Users' guides to the medical literature (XXIII. Qualitative research in health care. A. Are the results of the study valid?). J Am Med Assoc. 2000;284:357–362
  34. Giacomini MK, Cook DJ Evidence-Based Medicine Working Group. Users' guides to the medical literature (XXIII. Qualitative research in health care. B. What are the results and how do they help me care for my patients?). J Am Med Assoc. 2000;284:478–482
  35. Richardson WS, Wilson MC, Williams JW, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XXIV. How to use an article on the clinical manifestations of disease). J Am Med Assoc. 2000;284:869–875
  36. Guyatt GH, Haynes RB, Jaeschke RZ, et al. Evidence-Based Medicine Working Group Users' guides to the medical literature (XXV. Evidence-based medicine: Principles for applying the users' guides to patient care). J Am Med Assoc. 2000;284:1290–1296
  37. Malone DE. Evidence-based practice in radiology: An introduction to the series. Radiology. 2007;242:12–14
  38. Staunton M. Evidence-based radiology: Steps 1 and 2—Asking answerable questions and searching for evidence. Radiology. 2007;242:23–31
  39. Dodd JD. Evidence-based practice in radiology: Steps 3 and 4—Appraise and apply diagnostic radiology literature. Radiology. 2007;242:342–354
  40. Maher MM, Hodnett PA, Kalra MK. Evidence-based practice in radiology: Steps 3 and 4—Appraise and apply interventional radiology literature. Radiology. 2007;242:658–670
  41. Halligan S, Altman DG. Evidence-based practice in radiology: Steps 3 and 4—Appraise and apply systematic reviews and meta-analyses. Radiology. 2007;243:13–27
  42. Malone DE, Staunton M. Evidence-based practice in radiology: Step 5 (evaluate)—Caveats and common questions. Radiology. 2007;243:319–328

 This work was funded in part by GE-AUR Radiology Research Academic Fellowship.

PII: S0037-198X(09)00031-5

doi:10.1053/j.ro.2009.03.018

Seminars in Roentgenology
Volume 44, Issue 3 , Pages 209-213, July 2009

Article Tools