AI-P03 — AI for Think Tanks and Strategic Analysis

Course Code: AI-P03  |  School: School of Artificial Intelligence  |  Cluster: Level 3 — Applied & Sectoral Practice

Level: Intermediate  |  Duration: 5 weeks · 18–22 learning hours  |  Language: English  |  Certificate: Professional Certificate (non-degree)  |  Format: Self-paced with AI support under human supervision

Overview

A think tank sells judgement. Its output is valuable to the extent that a decision-maker can rely on it more than on their own reading, and that reliance rests on a reputation that takes years to build and one fabricated citation to destroy. This course is about using machine assistance to increase analytic throughput without touching the part of the product that carries the reputation.

The division taught here is between the labour of analysis and the judgement of analysis. Corpus assembly, translation, monitoring, first-pass summarisation of primary documents and the mechanics of formatting are labour. Framing the question, weighing contradictory evidence, assigning confidence, and deciding what a decision-maker most needs to know are judgement. Machine assistance is applied without restraint to the first and not at all to the second.

Learners produce a full analytic product to publication standard, supported by structured technique, an explicit confidence statement, a source verification appendix and a record of where assistance was used.

Learning outcomes

On completion, a successful learner will be able to:

1. Separate the labour and judgement components of an analytic task and allocate machine assistance accordingly.
2. Assemble and maintain a monitored source corpus, including in languages you do not read, while recording provenance.
3. Apply at least two structured analytic techniques to a live question, including one that generates competing explanations.
4. Express confidence and uncertainty in language that a decision-maker can act on and that can later be scored.
5. Verify every factual claim and citation in a published product, and evidence that verification.
6. Write a product that answers the question the decision-maker actually has, in the length they will actually read.

Who this course is for

Analysts and researchers in think tanks, policy institutes and research centres; strategic analysis units in government and industry; journalists producing analytical work; and doctoral researchers moving toward policy-facing output.

Prerequisites

Experience producing analytical writing for an external audience. AI-F06 recommended for the verification discipline assumed here.

Syllabus

Module 1 — Labour and judgement

Focus. Decomposing the analytic workflow and deciding, task by task, where assistance is legitimate. The specific danger of allowing assistance to migrate from summarising evidence to weighing it.

Lessons. 1.1 Decomposing the workflow. 1.2 Where assistance is legitimate. 1.3 Migration from summary to judgement. 1.4 What your reputation actually rests on.

Core reading. Richards J. Heuer Jr., Psychology of Intelligence Analysis (Washington, DC: Center for the Study of Intelligence, CIA, 1999), chapters 1–3.

Deliverable. Workflow decomposition for your own analytic product line, with assistance boundaries marked.

Module 2 — Corpus, monitoring and provenance

Focus. Building a durable source base rather than searching afresh each time. Multilingual sources and the risks of machine translation for evidentiary use. Recording provenance so a claim can be traced years later.

Lessons. 2.1 Designing a source corpus. 2.2 Monitoring and change detection. 2.3 Machine translation and evidentiary reliability. 2.4 Provenance records that survive staff turnover.

Core reading. Robert M. Clark, Intelligence Analysis: A Target-Centric Approach, 6th edition (Thousand Oaks: SAGE/CQ Press, 2020), chapters on source evaluation.

Deliverable. Source corpus specification with provenance schema and a monitoring plan.

Module 3 — Structured technique

Focus. Applying structured analytic techniques to counteract the specific biases that machine assistance intensifies: anchoring on a fluent first draft and premature closure. Analysis of competing hypotheses and key assumptions check.

Lessons. 3.1 Why structure beats effort. 3.2 Analysis of competing hypotheses. 3.3 Key assumptions check. 3.4 Using a model as a devil’s advocate without believing it.

Core reading. Randolph H. Pherson & Richards J. Heuer Jr., Structured Analytic Techniques for Intelligence Analysis, 3rd edition (Thousand Oaks: SAGE/CQ Press, 2020), chapters 7–8.

Deliverable. Completed competing hypotheses matrix and key assumptions check for a live question.

Module 4 — Confidence, calibration and the record

Focus. Distinguishing confidence in a judgement from the strength of evidence for it, expressing both in standard language, and keeping a record that allows your own accuracy to be scored over time.

Lessons. 4.1 Confidence versus evidence strength. 4.2 Standard probabilistic language. 4.3 Building a scoreable record. 4.4 Reporting when you do not know.

Core reading. Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton: Princeton University Press, 2005), chapters 2–3. Philip E. Tetlock & Dan Gardner, Superforecasting (New York: Crown, 2015), chapters 4 and 10.

Deliverable. Confidence statement and calibration record design for your unit.

Module 5 — Product, verification and publication

Focus. Writing for the decision-maker rather than for peers, the verification appendix, disclosure of assistance, and the institutional consequence of publishing an error.

Lessons. 5.1 The question the decision-maker actually has. 5.2 Length, structure and the executive judgement first. 5.3 Verification appendix. 5.4 Correction policy and reputational recovery.

Core reading. Andrew Rich, Think Tanks, Public Policy, and the Politics of Expertise (Cambridge: Cambridge University Press, 2004), chapters 1 and 5, on the basis of think-tank credibility.

Deliverable. Final submission: publication-standard analytic product with verification appendix and assistance disclosure.

Assessment

Pass mark 70 per cent. All assessed components must be attempted. Every mark in this course is issued by a human assessor; no assessment outcome is generated automatically.

Rubric criteria

Each assessed artefact is marked against four criteria at four levels (distinction, pass with merit, pass, fail).

1. Judgement retention: is the analytic judgement demonstrably the author’s, with assistance confined to labour?
2. Structural rigour: are competing explanations genuinely tested rather than listed and dismissed?
3. Calibration quality: is confidence expressed in a form that could later be scored against outcome?
4. Verification completeness: is every factual claim and citation traceable and verified?

Reading list

Core. Richards J. Heuer Jr., Psychology of Intelligence Analysis (Washington, DC: Center for the Study of Intelligence, CIA, 1999). Randolph H. Pherson & Richards J. Heuer Jr., Structured Analytic Techniques for Intelligence Analysis, 3rd edition (Thousand Oaks: SAGE/CQ Press, 2020).

Judgement and forecasting. Philip E. Tetlock, Expert Political Judgment (Princeton: Princeton University Press, 2005). Philip E. Tetlock & Dan Gardner, Superforecasting (New York: Crown, 2015).

Institutional context. Andrew Rich, Think Tanks, Public Policy, and the Politics of Expertise (Cambridge: Cambridge University Press, 2004). Thomas Medvetz, Think Tanks in America (Chicago: University of Chicago Press, 2012). Robert M. Clark, Intelligence Analysis: A Target-Centric Approach, 6th edition (Thousand Oaks: SAGE/CQ Press, 2020).

All items are published works identifiable by author, title and publisher. Learners obtain them through an institutional library or the publisher. The Academy does not distribute copyrighted texts.

Academic integrity and use of AI

Generative tools may be used in producing assessed work under three conditions. Use must be disclosed in a short statement appended to each submission, naming the tool and the task it performed. Any factual or technical claim originating from a generative tool must be verified against a citable source before it enters assessed work, and the verification must be evidenced. The analytical judgement in each artefact must be the learner’s own and must be defensible in a short follow-up. Products submitted in this course are treated as though they were published under your institution’s name. A single unverifiable claim fails the submission.

Instructor: pending owner confirmation. Pricing: pending owner approval. Reference list verified against publisher records; any later addition is marked for verification before publication.