Add cohort-specific study plans, prerequisite extraction and two new programmes

.github/workflows/ci.yml

@@ -64,5 +64,15 @@ jobs:
         # Fails when the brotlied client JS exceeds the budget in
         # `package.json` → `size-limit`. Bump the limit deliberately when an
         # increase is justified (new feature, dep upgrade); the diff documents
-        # why. The current budget has ~27% headroom over the baseline.
+        # why.
+        #
+        # The budget now covers two things that grow independently, because
+        # production chunk names are fully hashed and size-limit cannot tell
+        # them apart:
+        #   ~201 kB  application JS
+        #   ~50 kB   the committed cohort archive (src/data/cohorts/*.json),
+        #            ~2 kB brotlied per cohort file
+        # Each cohort file is its own lazily-loaded chunk, so an unselected
+        # cohort costs a viewer nothing — but every one counts here. When this
+        # fails, check `du -sh src/data/cohorts` before assuming the app grew.
         run: npm run size

README.md

@@ -46,9 +46,15 @@ Other hosts: Netlify or static exports (with limitations). See Next.js docs for
 
 Top-level (inside this folder):
 
-- `package.json` — project manifest and scripts (dev, build, start).
+- `package.json` — project manifest and scripts (dev, build, start, lint, validate-data, extract-plan, size).
 - `tsconfig.json` — TypeScript configuration.
 - `next.config.ts` — Next.js configuration.
+- `STUDY-PLAN-CONVENTIONS.md` — observations on how study plans differ between KTH schools, gathered while building the extractor.
+
+Scripts
+
+- `scripts/extract-from-kopps.mjs` — builds a cohort's study plan from KTH's published data. Pure Node, no extra dependencies. See `CLAUDE.md` for the full picture and `--help` for options.
+- `scripts/validate-data.mjs` — schema and cross-reference checks for everything in `src/data`, including the per-period full-time load check. Runs in CI.
 
 Key source files
 
@@ -63,8 +69,9 @@ Key source files
 
 Data files (rendered at runtime)
 
-- `src/data/programs.json` — list of available programs with their display names, optional inriktningar (specializations), and a study-plan URL.
-- `src/data/<PROGRAM>.json` — program-specific course datasets (currently CTFYS, CTMAT, CFATE, COPEN, CINEK, TIEMM). Each course includes fields such as `code`, `name`, `totalCredits`, `periodCredits` (P1–P4, either flat per-year or by-year for multi-year courses), `year`, `prerequisites`, and the `exams`/`reexams` arrays. Programs may also include `optionGroup` entries for course-choice slots.
+- `src/data/programs.json` — list of available programs with their display names, optional inriktningar (specializations), and a study-plan URL. Two flags control visibility: `verified: false` hides a plan behind the "show unverified" checkbox, while `disabled: true` withdraws it from the UI entirely (not in the dropdown, not reachable via `?program=`).
+- `src/data/<PROGRAM>.json` — program-specific course datasets (CTFYS, CTMAT, CFATE, COPEN, CINEK; CMAST and CMATD load a cohort file directly). Each course includes fields such as `code`, `name`, `totalCredits`, `periodCredits` (P1–P4, either flat per-year or by-year for multi-year courses), `year`, `prerequisitesCompleted` / `prerequisitesParticipation`, and the `exams`/`reexams` arrays. Programs may also include `optionGroup` entries for course-choice slots and `electivePlaceholder` entries for the space reserved for valfria kurser.
+- `src/data/cohorts/<PROGRAM>-HT<year>.json` — per-admission-cohort study plans, one file per cohort, each opening with a `cohortMeta` entry that records where every study year's data came from. Selected in the UI via `?cohort=HT2023`. `cohorts/index.json` lists what is available per program and is regenerated from disk by the extractor.
 - `src/data/<PROGRAM>-cosmetics.json` — per-course color-family assignments (capped at 5 families).
 - `src/data/kth-colors.json` — KTH color palette used for fills/strokes in the visualization.
 - `src/data/academic-periods.json` — academic period definitions (P1–P4) with `start`, `end`, `examStart`, `examEnd`, `reExamStart`, `reExamEnd` as ISO date strings. These are converted to Date objects in `src/types/course.ts`.
@@ -97,13 +104,17 @@ Exam/re-exam markers
 
 **Bilingual UI** (Swedish/English): Toggle from the export menu. The choice is reflected in the URL (`?l=sv` or `?l=en`).
 
-**Shareable URL state**: The selected program, language, hidden layers, and option-group picks are mirrored into the query string so a particular view can be linked or bookmarked.
+**Shareable URL state**: The selected program, admission cohort, language, hidden layers, specialization filter, and option-group picks are mirrored into the query string so a particular view can be linked or bookmarked — e.g. `?program=CTFYS&cohort=HT2023&l=sv`.
+
+**Cohort view**: A student picks their admission year and sees the plan as they will study it, rather than one calendar läsår sliced across three cohorts. Because KTH publishes only the läsår currently being taught and the next one, years missing for a cohort are borrowed from the nearest cohort that has them; a line under the selectors says which years are approximated, with per-year detail and a link to KTH's published plan behind an info affordance.
+
+**Elective space**: Where a period falls short of full-time (15 hp), the plan shows a "Plats för valfri kurs" placeholder sized to the shortfall, in yellow.
 
 **Specializations (inriktningar) and option groups**: Programs that split their bachelor years by inriktning (e.g. CINEK) get a SpecializationFilter UI that AND-filters courses across spec groups. Course-choice slots (e.g. thesis-track options) are modelled as `optionGroup` entries and rendered with a striped pattern; clicking opens a selection modal.
 
 Troubleshooting
 
-- Port 3000 already in use: find and kill the process `lsof -iTCP:3000 -sTCP:LISTEN -n -P` then `kill <PID>`.
+- Port 3000 already in use: check what is listening with `lsof -iTCP:3000 -sTCP:LISTEN -n -P`, or just start on another port with `npm run dev -- --port 3100`. Note that on macOS a second server can bind `*:3000` while an existing one holds `[::1]:3000` — both appear to start, but `localhost:3000` reaches the first one.
 - Suspended dev job (Ctrl+Z): resume with `fg` or start a background server with `nohup` as shown above.
 - Type errors: run `npx tsc --noEmit` to see TypeScript diagnostics.
 - PDF export not working on Vercel: Ensure `vercel.json` is deployed with the project and `@sparticuz/chromium` is in dependencies.
@@ -114,17 +125,20 @@ Key production dependencies:
 - `next` (16.x) — React framework
 - `react` (19.x) — UI library
 - `d3` (7.9.x) — Visualization and data manipulation
-- `puppeteer-core` (23.x) — Headless browser control for PDF generation
+- `puppeteer-core` (25.x) — Headless browser control for PDF generation
 - `@sparticuz/chromium` (141.x) — Serverless-compatible Chrome binary for Vercel
 
+The study-plan extractor deliberately adds no dependencies — it runs on plain Node
+with `fetch`, so it works in a checkout with no `node_modules` at all.
+
 Development dependencies include TypeScript, ESLint, and Tailwind CSS.
 
 ## Configuration Files
 
 - `vercel.json` — Vercel deployment configuration with increased memory (1800MB for Hobby plan, can be increased to 3008MB on Pro) and timeout (60s) for the PDF export API route.
 - `tsconfig.json` — TypeScript configuration.
 - `next.config.ts` — Next.js configuration.
-- `package.json` — Project manifest and scripts (dev, build, start).
+- `package.json` — Project manifest and scripts (dev, build, start, lint, validate-data, extract-plan, size). The `size-limit` budget covers every static chunk, which includes the committed cohort archive — see the note in `.github/workflows/ci.yml` before assuming a failure means the app grew.
 
 ## License & data attribution
 

STUDY-PLAN-CONVENTIONS.md

@@ -0,0 +1,206 @@
+# How KTH study plans are written: observed differences between programmes
+
+KTH schools run different kinds of programme, so some of what follows is a
+legitimate consequence of different needs — a two-year master's programme has
+little reason to describe elective space the way a five-year civilingenjör
+programme does. Still, several of the differences below look incidental rather
+than deliberate: the same fact is recorded in different places, or in prose in
+one programme and in structured fields in another. Where that is the case, one
+convention is usually easier to read, to maintain and to consume programmatically
+than the other, and it seems worth comparing notes across schools.
+
+These are observations, not conclusions. They were gathered while building a tool
+that renders utbildningsplaner from KTH's published data (see `CLAUDE.md`), so the
+lens is deliberately narrow: what is machine-readable, what is consistent, and
+what needs a human to interpret. Anyone who owns one of these programmes will
+know better than we do whether a difference is meaningful.
+
+Everything is dated and sourced so it can be re-checked. Figures were measured in
+August 2026 over eight programmes: **CTFYS, CTMAT, CFATE, COPEN** (owned by SCI,
+Teknikvetenskap) and **CINEK, TIEMM, CMAST, CMATD** (owned by ITM, Industriell
+teknik och management).
+
+---
+
+## 1. Elective space: stated as a number in some programmes, only in prose in others
+
+Full-time study is 15 hp per period, so a year's courses should add up to 15 in
+every period. Where they do not, the gap is normally the space for *valfria
+kurser*. That space is real, and students need to see it — but how it is recorded
+varies:
+
+| programme | how elective space is discoverable |
+|---|---|
+| CTMAT | prose, **with a figure**: *"Utrymmet för valfria kurser är 7,5 hp per period hela läsåret."* |
+| CTFYS | prose, **with a figure**: *"På våren i årskurs 3 finns ett utrymme på 15,0 hp valfria kurser."* |
+| TIEMM | prose, **no figure**: *"…kan du även välja några helt valfria kurser."* |
+| CFATE | not stated; the gap has to be inferred from the credit arithmetic |
+| CMAST | not stated; a uniform 4.5 hp gap in year 2 P3 and P4 across all inriktningar |
+
+The two programmes that give a figure are the two where the space can be filled
+in automatically and checked — the stated amount matched the computed shortfall in
+both cases. Where no figure is given, the arithmetic still reveals *that* space
+exists, but not how it is meant to be distributed.
+
+**Suggestion.** Stating elective space as a number, per period, is materially
+more useful than stating it qualitatively — and more useful still would be
+expressing it as an entry in the plan rather than as prose, so it does not have to
+be recovered by subtraction. The wording CTMAT uses reads well and is
+unambiguous: *"Utrymmet för valfria kurser är N hp per period."*
+
+## 2. "Villkorligt valfri" is used for two different things
+
+`electiveCondition: VV` marks a course as villkorligt valfri. Two distinct
+situations are recorded the same way:
+
+- **A genuine choice.** CTFYS year 3 lists `EF112X` and `SA114X` — pick one, both
+  15 hp in P3+P4. This is a clean pick-one group and renders as one.
+- **A pool with a credit threshold.** CFATE year 3 lists 36 hp of obligatorisk
+  courses plus a 15 hp thesis choice, leaving 3 hp of the 60 — but then lists five
+  villkorligt valfria courses totalling 26 hp. The student takes a subset. The
+  intent is "minst N hp ur denna grupp", which the VV flag cannot express.
+
+The second case is not detectable from the flag alone; it only shows up as a
+period summing past full-time. CMAST year 3 shows the same shape for several
+inriktningar.
+
+**Suggestion.** Where the requirement is "at least N hp from this set", recording
+the threshold alongside the set would remove the ambiguity. As published, a
+consumer cannot distinguish "choose one of these" from "choose 3 hp worth of
+these" without doing credit arithmetic and guessing.
+
+## 3. A "group" of one
+
+15 of 34 option groups extracted across the programmes contained exactly **one**
+course — 5 of CFATE's 7 and 10 of TIEMM's 23. Kopps marks these villkorligt
+valfri, but a choice between one alternative is not a choice, and none of the six
+hand-curated study plans in this project models them as groups.
+
+This is probably an artefact of how the plan was entered rather than an intent.
+Worth a look by whoever maintains those plans: either the group is missing its
+other members, or the course is effectively obligatorisk within that inriktning.
+
+## 4. Structure splits by school; wording does not
+
+Two things vary independently, and it is useful to keep them apart.
+
+**Structure follows the owning school.** The SCI programmes return a single
+curriculum with no inriktningar in years 1–3. The ITM programmes split by
+inriktning from year 2:
+
+| programme | school | inriktningar in years 1–3 |
+|---|---|---|
+| CTFYS, CTMAT, CFATE, COPEN | SCI | 0 |
+| CINEK | ITM | 4 |
+| CMATD | ITM | 4–6 |
+| CMAST | ITM | 3 in year 2, **14** in year 3 |
+
+Both conventions are defensible — the SCI programmes genuinely do not branch
+before the master's choice. But the difference means a consumer must handle both,
+and CMAST year 3 (15 parallel curriculum variants) is an order of magnitude more
+complex than anything on the SCI side.
+
+**Wording does not follow the school**, because prerequisite text belongs to the
+*course*, not the programme: 23 courses appear in both an SCI and an ITM programme
+and carry identical text in each. The variation tracks the department that owns
+the course and the cycle level. TIEMM, a master's programme, is the outlier on
+every measure (80 % of its prerequisite texts use "motsvarande", against 15–29 %
+for the bachelor programmes).
+
+## 5. `electiveCondition: R` is used by one programme only
+
+CMAST records 144 participations with `electiveCondition: R` (rekommenderad) —
+courses recommended for the master track a given inriktning leads to. None of the
+other seven programmes uses this value at all.
+
+It is genuinely useful information: it answers "what should I put in my elective
+slots?", which none of the other programmes answers in structured form. But
+because only one programme uses it, a consumer that has not met CMAST will not
+know the value exists.
+
+**Suggestion.** This looks like the better convention, not the deviant one. If
+recommended-course-per-track were recorded this way across programmes, the
+"elective space" problem in §1 would largely solve itself: the space and the
+suggested ways to fill it would both be structured data. Worth discussing whether
+CMAST's practice should spread rather than be normalised away.
+
+## 6. Prerequisites: free text carrying structured intent
+
+Prerequisites are published only as prose, in the syllabus *Särskild behörighet*
+field, while what they express is almost always structured: a set of courses and a
+requirement type. Recurring patterns:
+
+```
+"Aktivt deltagande i SF1673 Analys i en variabel."               -> participation
+"Slutförd kurs SF1672 Linjär algebra"                            -> completed
+"SG1112 Mekanik I eller motsvarande"                             -> completed, type implicit
+"Kunskaper … motsvarande slutförd kurs DD1310-DD1319/DD1331/…"    -> a long alternative list
+"…slutfört moment LAB1 i SH1017"                                 -> one module of another course
+"Minst 104 högskolepoäng … ska vara avklarade"                    -> a credit threshold
+```
+
+Four specific frictions, each of which needed a rule to work around:
+
+1. **Type is often implicit.** "SG1112 Mekanik I eller motsvarande" does not say
+   whether the course must be completed or merely attended. The distinction
+   matters — one must finish before the course starts, the other may run in
+   parallel.
+2. **Requirements are not reliably separated.** EI1320's 2026 syllabus runs two
+   requirements together with no punctuation between them: *"…motsvarande slutförd
+   kurs SI1200 eller SF1693 Kunskaper i grundläggande elektromagnetism…,
+   motsvarande slutförd kurs SK1104/SH1017…"*. Read as one clause, two independent
+   requirements look like alternatives.
+3. **Alternative sets are written two ways** — slash lists (`DD1331/DD1337`) and
+   hyphen ranges (`DD1310-DD1319`) — sometimes both in one sentence. 43 range
+   expressions appear across the eight programmes.
+4. **Lists of qualifying courses go stale.** DD1385 and DD1380 ask for knowledge
+   in programming and list `DD1310/DD1311/…/DD1331` — but not `DD1333`, which is
+   CTMAT's own first-year programming course. Neither syllabus has been revised
+   since HT2021. DD1328 had the same omission and it *was* fixed in its 2026
+   revision, which shows the process works when someone notices.
+
+**Suggestion.** Point 4 is the one with a clear owner: when a course lists the
+courses that satisfy a knowledge requirement, that list needs revisiting whenever
+a programme introduces a new course covering the same ground. Points 1–3 are
+about wording, and a short house style would help — always state "slutförd" or
+"aktivt deltagande" explicitly, and start each requirement as its own sentence.
+
+## 7. Where the same fact lives in different places
+
+Smaller observations, each costing a consumer a special case:
+
+- **Course period data is authoritative in one place and stale in another.** The
+  KOPPS API returns an older syllabus version than the course page for 51 of 217
+  courses checked. Since KOPPS is being retired this is expected, but it is worth
+  saying explicitly somewhere public that it should no longer be read.
+- **A programme with no curriculum for a year says so in prose.** CFATE and CTFYS
+  both explain in `supplementaryInformation` that years 4–5 are taken inside a
+  master's programme. COPEN has no year 2–3 curriculum at all, and that is only
+  discoverable by finding zero courses listed.
+- **Known data errors are documented in prose.** CTMAT's plan says: *"En bugg gör
+  tyvärr att fel poängfördelning för SA1006 visas i studentgränssnittet."*
+  Honest and helpful to a human reader, invisible to anything automated.
+- **Selecting a historical course version is no longer possible in the UI.** The
+  `?startterm=` parameter is ignored; only the current round is shown. The full
+  version history is still present in the page's own data, and in PDF form in the
+  kursutveckling archive, but a programme director cannot browse to the version a
+  past cohort actually studied.
+
+---
+
+## Summary of what seems to work best
+
+Drawn from the above, and offered for discussion rather than as recommendations:
+
+| topic | the convention that reads best | seen in |
+|---|---|---|
+| elective space | stated as hp per period, in the plan rather than in prose | CTMAT (prose with figure) |
+| recommended courses | structured per inriktning, not described in text | CMAST (`R`) |
+| credit-threshold groups | threshold recorded with the set | nowhere yet |
+| prerequisite type | "slutförd" / "aktivt deltagande" always explicit | CTFYS, mostly |
+| requirement separation | one sentence per requirement | most, but not EI1320 |
+| qualifying-course lists | reviewed when a programme adds a covering course | DD1328 (fixed 2026) |
+
+None of this is urgent. But since several of these are already done well
+*somewhere* at KTH, harmonising is mostly a matter of picking whichever
+convention already exists rather than inventing anything.

package.json

@@ -7,15 +7,15 @@
     "build": "next build",
     "start": "next start",
     "lint": "eslint . --max-warnings 0",
-    "validate-data": "node scripts/validate-data.mjs",
+    "validate-data": "node scripts/validate-data.mjs --cohorts",
     "extract-plan": "node scripts/extract-from-kopps.mjs",
     "size": "size-limit"
   },
   "size-limit": [
     {
-      "name": "Client JS (brotlied, all static chunks)",
+      "name": "Client JS + cohort data (brotlied, all static chunks)",
       "path": ".next/static/chunks/**/*.js",
-      "limit": "250 KB"
+      "limit": "330 KB"
     }
   ],
   "dependencies": {