AI-F03 — How AI Systems Work: Models, Data, and Machine Learning Concepts

Course Code: AI-F03  |  School: School of Artificial Intelligence  |  Cluster: Level 1 — Foundations

Level: Introductory  |  Duration: 4 weeks · 12–16 learning hours  |  Language: English  |  Certificate: Certificate of Completion (non-degree)  |  Format: Self-paced with AI support under human supervision

Overview

This course explains how contemporary AI systems actually produce their outputs, for people who will never build one but who must judge one. It covers the mechanics of learning from data, the difference between a model and a program, and the specific reasons a statistical system can be highly capable and confidently wrong at the same time.

The teaching approach is conceptual rather than mathematical. There is no coding and no calculus. What replaces them is precision of language: learners finish able to say what training, inference, parameters, embeddings, fine-tuning and evaluation mean, and to notice when a vendor or a colleague is using those words loosely. That precision is the practical skill, because almost every bad institutional decision about AI traces back to a category error about what the system is doing.

The course closes on evaluation. Learners are taught to ask what a claimed accuracy figure was measured on, what the baseline was, and what population the test set represented, because a benchmark number without those three answers carries no information.

Learning outcomes

On completion, a successful learner will be able to:

1. Explain in plain language the difference between rule-based software, supervised learning, and generative models trained on large corpora.
2. Describe how training data shapes model behaviour, and identify the properties of a dataset that predict downstream failure.
3. Account for why a language model produces fluent text that may be factually wrong, without resorting to metaphors of understanding or intent.
4. Interpret a performance claim critically, asking what was measured, against what baseline, and on which population.
5. Read a model card or datasheet and state what it does and does not disclose.
6. Judge whether a proposed use case is a reasonable fit for a statistical system or requires deterministic logic.

Who this course is for

Lecturers, researchers, civil servants, managers and analysts who evaluate, procure or supervise AI systems without building them. Also suitable as preparation for the governance and applied clusters of this school.

Prerequisites

No programming or mathematics background. Basic computer literacy and a willingness to read carefully. AI-F01 is recommended but not required.

Syllabus

Module 1 — Programs, models, and the shift from instruction to inference

Focus. Why a trained model is not a set of rules, and what changes when behaviour is learned from examples rather than specified by a programmer. Parameters, training, inference, and the meaning of generalisation.

Lessons. 1.1 Deterministic software and its guarantees. 1.2 Learning from examples. 1.3 Training and inference as separate events. 1.4 Generalisation and why it fails.

Core reading. Ethem Alpaydin, Machine Learning, revised and updated edition (Cambridge, MA: MIT Press, 2021), chapters 1–2.

Deliverable. One-page written explanation of a chosen system, distinguishing what was specified from what was learned.

Module 2 — Data as the real substance of the system

Focus. Where training data comes from, what it excludes, and how collection decisions become behavioural properties. Labels, sampling, distribution shift, and the documentation practices designed to make these visible.

Lessons. 2.1 Collection, labelling and their costs. 2.2 Sampling, coverage and who is missing. 2.3 Distribution shift after deployment. 2.4 Datasheets as accountability instruments.

Core reading. Timnit Gebru et al., “Datasheets for Datasets”, Communications of the ACM 64, no. 12 (2021): 86–92.

Deliverable. Completed datasheet-style questionnaire for a dataset relevant to your institution.

Module 3 — Language models: what fluency is and is not

Focus. Next-token prediction, scale effects, context windows, and the structural reason that fluent output carries no guarantee of accuracy. Fine-tuning, instruction tuning and retrieval as distinct interventions.

Lessons. 3.1 Prediction over meaning. 3.2 Scale, emergence and the limits of the claim. 3.3 Fine-tuning, instruction tuning, retrieval. 3.4 Why confident error is the default failure mode.

Core reading. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major & Shmargaret Shmitchell, “On the Dangers of Stochastic Parrots”, Proceedings of FAccT (ACM, 2021). Rishi Bommasani et al., On the Opportunities and Risks of Foundation Models (Stanford CRFM, 2021), sections 1 and 4.

Deliverable. Annotated transcript of a model interaction identifying three failure types and their probable causes.

Module 4 — Evaluation, disclosure, and reading a claim

Focus. Benchmarks and their limits, baselines, test-set contamination, and the difference between a demonstration and an evaluation. Model cards as the disclosure instrument, and what they typically omit.

Lessons. 4.1 What a benchmark measures. 4.2 Baselines and the missing comparison. 4.3 Contamination and overfitting to the test. 4.4 Model cards: reading the gaps.

Core reading. Margaret Mitchell et al., “Model Cards for Model Reporting”, Proceedings of FAT* (ACM, 2019).

Deliverable. Critical appraisal (two pages) of one published performance claim for a system your institution might adopt.

Assessment

Pass mark 70 per cent. All assessed components must be attempted. Every mark in this course is issued by a human assessor; no assessment outcome is generated automatically.

Rubric criteria

Each assessed artefact is marked against four criteria at four levels (distinction, pass with merit, pass, fail).

1. Conceptual accuracy: are technical terms used in their proper sense rather than metaphorically?
2. Evidential discipline: are claims about a system supported by documentation or observation rather than assumption?
3. Critical reading: does the appraisal identify what a source does not say as well as what it does?
4. Communicability: could a non-specialist colleague act on the written output?

Reading list

Core. Ethem Alpaydin, Machine Learning, revised and updated edition (Cambridge, MA: MIT Press, 2021). Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans (New York: Farrar, Straus and Giroux, 2019).

Reference. Stuart Russell & Peter Norvig, Artificial Intelligence: A Modern Approach, 4th edition (Hoboken: Pearson, 2021), chapters 1 and 19 for orientation only.

Peer-reviewed. Timnit Gebru et al., “Datasheets for Datasets”, Communications of the ACM 64, no. 12 (2021). Margaret Mitchell et al., “Model Cards for Model Reporting”, Proceedings of FAT* (ACM, 2019). Emily M. Bender et al., “On the Dangers of Stochastic Parrots”, Proceedings of FAccT (ACM, 2021). Ziwei Ji et al., “Survey of Hallucination in Natural Language Generation”, ACM Computing Surveys 55, no. 12 (2023).

Report. Rishi Bommasani et al., On the Opportunities and Risks of Foundation Models (Stanford Center for Research on Foundation Models, 2021).

All items are published works identifiable by author, title and publisher. Learners obtain them through an institutional library or the publisher. The Academy does not distribute copyrighted texts.

Academic integrity and use of AI

Generative tools may be used in producing assessed work under three conditions. Use must be disclosed in a short statement appended to each submission, naming the tool and the task it performed. Any factual or technical claim originating from a generative tool must be verified against a citable source before it enters assessed work, and the verification must be evidenced. The analytical judgement in each artefact must be the learner’s own and must be defensible in a short follow-up. Because this course teaches how these systems fail, submissions that reproduce model output as explanation without independent verification will be returned unmarked.

Instructor: pending owner confirmation. Pricing: pending owner approval. Reference list verified against publisher records; any later addition is marked for verification before publication.