Analysis and Judgement - KBA13 Academy

PCS-C06 · Module 4 — Analysis · Lesson 4.1 · Study time: 75 minutes · Language: English

Learning objectives

By the end of this lesson you will be able to:

Move from a body of evidence about conflict assessment methods to a defensible judgement, and show the steps between.
Express uncertainty in language that a decision-maker can act on.
Identify and correct the biases that most often distort analysis in peace, conflict and security studies.
Write an analytical product that separates what is known, what is assessed and what is assumed.

Key terms

Judgement: A conclusion that goes beyond the evidence by a stated amount. Analysis that never exceeds the evidence is a summary; analysis that exceeds it without saying so is speculation.
Confidence level: A statement of how much weight the analyst places on a judgement, and why — usually a function of evidence quality, source independence and the stability of the underlying assumptions.
Assumption: A proposition treated as true for the purposes of the analysis without being demonstrated. Legitimate, provided it is labelled and its failure conditions are stated.
Anchoring: The tendency for an initial figure or framing to constrain all subsequent estimates, even when it is known to be arbitrary.
Mirror-imaging: Assuming other actors reason as you would, with your information, values and constraints. A recurrent source of large errors in peace, conflict and security studies.

Lesson

From evidence to judgement

This lesson. Analysis and Judgement belongs to Module 4 — Analysis of PCS-C06 Conflict Assessment Methods, taught in the School of Peace, Conflict and Security Studies. Work through the written lesson first, then the slide deck below, then the activity. Every element on this page is live HTML text: it can be enlarged, printed, searched, read aloud by assistive technology and machine-translated without loss.

The move from evidence to judgement is where most analytical work is won or lost, and it is the step most often left implicit. A reader given a body of material and a conclusion has no way of checking the inference unless the inference is written down. Good practice is to make the step explicit in one sentence of the form: given these three items of evidence, and assuming this condition holds, the most probable explanation of conflict assessment methods in this case is X, and it is more probable than Y because of a specific discriminating observation.

Notice that this formulation contains four separable parts: evidence, assumption, judgement and the comparison that makes the judgement preferable. A reader who disagrees can now say precisely where. That is the purpose. Analysis is not an attempt to be unarguably right; it is an attempt to be arguable in a productive way, so that the disagreement can be resolved by evidence rather than by seniority.

Expressing uncertainty

Verbal probability terms are treated by readers as far more precise than analysts intend. Research on this is consistent: words such as ‘likely’, ‘probable’ and ‘a real possibility’ are interpreted across enormous ranges by different readers, and the same reader will interpret them differently depending on the topic. The remedy is to use a fixed, published vocabulary and, where the analysis supports it, to attach numeric ranges. If your organisation has no such scale, state one at the top of your product.

Separate two things that are routinely confused: how likely the judgement is to be correct, and how confident you are in that estimate. A high-probability judgement resting on one unverified source and a moderate-probability judgement resting on several independent well-appraised sources are different products, and a decision-maker needs to know which they are holding. Say both, in that order, and give the reason for the confidence rating in half a sentence.

The biases that matter here

Four biases account for most avoidable error in conflict assessment methods. Anchoring binds later estimates to an early number. Confirmation bias makes disconfirming evidence feel weaker than it is, and it operates most strongly on people who are expert, because expertise supplies ready reasons to discount. Mirror-imaging assumes the other party shares your information and priorities. Availability makes recent and vivid cases dominate base rates that are more informative.

Awareness alone does not fix any of these; the literature is clear that knowing about a bias barely reduces it. Procedures do. Generate the alternative explanation before looking at the evidence rather than afterwards. Ask a colleague to argue the opposing case with real effort. Record your estimate before you see someone else’s. Apply Conflict transformation, which treats conflict as relational and long-term rather than as an incident to be settled. These are cheap, and their value comes precisely from the fact that they do not depend on your being unbiased in the moment.

Writing the product

An analytical product should let a busy reader stop after the first paragraph and still have the judgement, its confidence level and the main reason for it. Everything after that is support, in descending order of importance. Do not build to a conclusion; the structure of a good analytical brief is the inverse of the structure of a good essay, and students moving between the two forms routinely get this wrong.

Then keep three registers visibly separate throughout: what is known, what is assessed, and what is assumed. Use different sentence forms so the reader can tell them apart without effort — reported facts with sources attached, judgements marked with the probability vocabulary, assumptions flagged as such. A brief that blends the three reads more smoothly and is much more dangerous, because the reader cannot tell which parts would survive if one assumption failed.

Worked example: three sentences that carry a judgement

Consider a analyst asked to assess a developing situation in conflict assessment methods. A weak product opens: ‘The situation is complex and a number of factors are in play.’ It is true, it is safe, and it tells the reader nothing they did not know. A strong product opens with the judgement, the confidence and the reason, in that order, and does it in about forty words.

The strong version reads roughly: we assess it likely, with moderate confidence, that the pattern observed in conflict assessment methods will continue over the next two quarters; confidence is moderate because the two supporting reports share a common origin; this assessment would change if the underlying incentive structure were altered. Every element of that is checkable. A reader can act on it, can see exactly how much weight it will bear, and knows what to watch for. That is the whole craft, compressed.

Slide deck

A ten-slide presentation of this lesson, built in HTML so that it prints, translates, enlarges and reads aloud on any device.

Slide 1 of 10