COURSE.md

id	ai-from-first-principles
title	AI from First Principles
version	0.7.0-ai-101.1
summary	Learn, inspect, implement, and debug the foundations behind modern language models.
license	CC-BY-4.0
audience	software developers new to machine learning technical learners comfortable reading code
prerequisites	basic programming and terminal use ordinary arithmetic and arrays no prior machine-learning or calculus knowledge
estimatedHours	20
repository	https://github.com/Doppp/explorables
language	en
tags	machine-learning language-models
teaching	mode tutor-led
guidance	defaultMode allowExploreMode allowSkipping persistLocally guided true true true

AI from First Principles

This course assumes that you can read basic code, use arrays, and work in a terminal. It does not assume that you already know AI or machine-learning terminology. Each important term is introduced with a concrete example before a checkpoint asks you to use it. Predictions are starting guesses, not grades. If a word is unclear, stop and ask the tutor to explain that word before continuing.

Start with the words used to describe software and learned models. Then locate classification, generation, language models, chat products, training, inference, and reasoning before meeting the mathematical machinery. The coding agent teaches and adapts the active checkpoint in conversation. The browser is the adjacent workbench for predictions, manipulation, evidence, and durable reference notes.

What you will learn

You will build a language model from the operations beneath it: learning from examples, calculating updates, representing text as numbers, mixing information with attention, training a Transformer, and generating and evaluating outputs. Every new term is introduced before a checkpoint asks you to use it.

How to use the course

Ask the coding-agent tutor to introduce the active checkpoint. Answer its prediction in chat, use the browser explorable to generate evidence, then return to conversation to explain what happened. Open the browser's reference notes whenever you want the canonical definitions or worked example. Guided checkpoints remain local to this browser and are not grades.

Use the course glossary to revisit the canonical beginner definition of a term. The lesson that first uses a term still explains it in context, so the glossary is a review aid rather than a prerequisite reading assignment.

Lessons

1. Start with the words
2. Inputs, outputs, rules, and learned models
3. Generative AI and language models
4. The next-token loop
5. How machines learn
6. Inference, generation, and reasoning
7. Gradient descent
8. Backpropagation
9. Vectors, matrices, and linear layers
10. Losses and optimisers
11. BPE tokenisation
12. Embeddings and positional information
13. Self-attention
14. Multi-head attention
15. The Transformer block
16. Next-token training
17. Autoregressive inference and KV caching
18. Sampling and generation
19. Evaluation leakage