MED-04 · MODULE 7 · LESSON 7.1

Evidence, Guidelines, and Critical Appraisal

Estimated study time: 3–4 hours · Level: University (Harvard/Oxford/Yale standard) · Prerequisite: Modules 1–6 of Clinical Diagnostics and Laboratory Medicine

Intended Learning Outcomes

  1. Explain the hierarchy of clinical evidence and what each study design can and cannot establish.
  2. Appraise a research paper for internal validity, bias, and applicability.
  3. Interpret guideline recommendations in light of their supporting evidence and grade.
  4. Evaluate the role of confounding, chance, and effect size in clinical claims.
  5. Apply appraisal skills to decide whether a finding should change practice.

1. The Evidence Hierarchy

Not all evidence is equal. Systematic reviews and well-conducted randomised controlled trials sit near the top of the conventional hierarchy, while case reports and expert opinion sit lower. This section explains the logic of the hierarchy and, importantly, its limits.

The hierarchy is a heuristic, not a law. A superbly conducted cohort study can outweigh a small, biased trial. The skilled reader judges the study, not merely its label.

2. Bias, Confounding, and Chance

Three threats stand between data and truth: systematic bias, confounding, and random error. We define each, show how study design mitigates them, and illustrate the residual risks that survive even good design.

Confidence intervals and p-values are examined as tools for quantifying chance, together with their frequent misinterpretation.

3. Reading Guidelines Critically

Clinical guidelines translate evidence into recommendations, but the strength of a recommendation and the quality of its evidence are separate axes. We use the GRADE framework to read recommendations with appropriate scepticism.

A strong recommendation on weak evidence, or a weak one on strong evidence, both carry meaning that the careful reader must decode.

4. From Appraisal to Action

The purpose of appraisal is decision. We close with a structured approach to deciding whether a given finding is valid, important, and applicable to the patient or setting in front of you.

This mirrors the evidence-based-medicine tradition taught at leading schools: ask, acquire, appraise, apply, assess.

Key Terminology

TermDefinition
RCTRandomised controlled trial — allocation by chance to reduce confounding.
Systematic reviewA structured synthesis of all eligible studies on a question.
ConfounderA factor associated with both exposure and outcome that distorts their apparent link.
Confidence intervalA range of values compatible with the data at a stated level of confidence.
GRADEA system for rating the certainty of evidence and the strength of recommendations.
NNTNumber needed to treat — patients treated for one to benefit.
Interactive Concept Map
Why is a hierarchy of evidence useful?
It gives a first-pass estimate of how much confidence a design can support, guiding where to look and how hard to scrutinise.
When does the hierarchy mislead?
When study quality within a tier varies, or when the clinical question is not one a trial can ethically or practically answer.
What is the difference between bias and chance?
Bias is a systematic distortion that larger samples do not fix; chance is random error that shrinks as samples grow.
How do I read a confidence interval?
Look at whether it crosses the no-effect value and how wide it is — width reflects precision, position reflects direction and size.
What does GRADE add?
It separates the certainty of evidence from the strength of a recommendation, making both explicit.
When should a finding change practice?
When it is valid, the effect is important, and it applies to your patients — all three, not one.
💡 Big idea: Evidence quality and recommendation strength are separate axes; appraisal means judging both.
Interactive Prototype — Step Through the Reasoning
Step 1: Ask a focused question
Frame it in PICO terms: population, intervention, comparator, outcome.
Step 2: Acquire the best evidence
Search systematically, favouring syntheses and high-quality primary studies.
Step 3: Appraise validity
Check for bias, confounding, and adequate power before believing the result.
Step 4: Assess importance
Look at effect size, confidence intervals, and NNT — statistical significance is not clinical importance.
Step 5: Apply to the patient
Ask whether your patient resembles the study population and whether values and circumstances align.
🎯 Take-home: Ask, acquire, appraise, apply, assess — the five-step evidence-based cycle.

Worked Example

Scenario. A new trial reports a statistically significant benefit for a therapy.

Appraise: was allocation randomised and concealed, follow-up complete, analysis intention-to-treat? Assess: is the confidence interval narrow and the effect size clinically meaningful, or is a trivial effect merely significant because the sample was huge? Apply: does your patient resemble the enrolled population? Only when all three pass does the trial warrant a change in practice.

Practice & Formative Check

  1. Take a recent paper and classify its design within the evidence hierarchy, justifying the placement.
  2. Identify one plausible confounder in an observational study you have read and explain how it could distort the result.
  3. Find a guideline recommendation and separate its recommendation strength from its evidence certainty.
Self-check: Can I state the five EBM steps?
Ask, acquire, appraise, apply, assess. If you can define each, you have met the outcome.
Self-check: Do I distinguish bias from chance?
Bias is systematic and unfixed by sample size; chance shrinks with larger samples. Be able to give an example of each.
Self-check: Can I read a confidence interval?
You should interpret width as precision and position relative to no-effect as direction and significance.

Reading List & References

Core textbooks

  • Guyatt G et al. — Users’ Guides to the Medical Literature (JAMA/McGraw-Hill), latest edition.
  • Straus S et al. — Evidence-Based Medicine: How to Practice and Teach EBM, latest edition.
  • Greenhalgh T — How to Read a Paper: The Basics of Evidence-Based Medicine, latest edition.

Applied & reference

  • The GRADE Handbook (GRADE Working Group) — open access.
  • CONSORT, STROBE, and PRISMA reporting guidelines (EQUATOR Network).
  • Cochrane Handbook for Systematic Reviews of Interventions — latest version.

Open & primary sources

  • Cochrane Library — open-access systematic review abstracts.
  • PubMed Central — open-access primary trials and reviews.
  • BMJ Best Practice / NICE guidance summaries where available.
KBA13 Academy · MED-04 Module 7. Prepared to a Harvard/Oxford/Yale university standard. Consult all sources in their most recent editions.

Slide deck

A ten-slide HTML presentation summarising this lesson. It prints, translates and reads on any device.

A ten-slide presentation of this lesson. Every slide is plain HTML, so it prints, translates and reads on any device.

Slide 1 of 10

Evidence in Clinical Diagnostics and Laboratory Medicine

Where the evidence comes from, what it can establish and how it fails.

Module 3 of MED-04.

MED-04 · Lesson 3.2KBA13 Academy

Slide 2 of 10

Classes of evidence

  • randomised controlled trials
  • cohort and case-control studies
  • systematic reviews and meta-analyses
  • clinical practice guidelines

Each is produced by an institution with its own purposes.

MED-04 · Lesson 3.2KBA13 Academy

Slide 3 of 10

Further classes

  • registry and audit data
  • laboratory and imaging findings

Not interchangeable. Different classes have different blind spots.

MED-04 · Lesson 3.2KBA13 Academy

Slide 4 of 10

Triangulation

Take a claim established in one evidence class and test it against another with different blind spots.

Where they diverge, the divergence is the finding.

MED-04 · Lesson 3.2KBA13 Academy

Slide 5 of 10

Four separate appraisal questions

  • Provenance: who made it, when, why?
  • Reliability: the source’s record over time
  • Credibility: this specific report
  • Relevance: the question you are actually asking

MED-04 · Lesson 3.2KBA13 Academy

Slide 6 of 10

Circular sourcing

Five reports; four of them citing the fifth.

Follow every citation chain to where it terminates.

MED-04 · Lesson 3.2KBA13 Academy

Slide 7 of 10

Measurement drift

The indicator was redefined; the world may not have changed at all.

Check the instrument before believing the series.

MED-04 · Lesson 3.2KBA13 Academy

Slide 8 of 10

Survivorship and timing

  • Records describe cases that passed an earlier filter
  • Retrospective accounts are contaminated by the outcome
  • Absence of a case is not absence of the phenomenon

MED-04 · Lesson 3.2KBA13 Academy

Slide 9 of 10

Recording standard

  • Full citation and date accessed
  • Specific page or section
  • The claim it supports
  • Your appraisal and your residual doubt

MED-04 · Lesson 3.2KBA13 Academy

Slide 10 of 10

Cite what you used

If you read it in a review, cite the review and say so.

Concealed second-hand citation transfers unearned authority and propagates error.

MED-04 · Lesson 3.2KBA13 Academy

12345678910

Click a number to jump to a slide, or scroll the deck sideways.

PreviousNext