Commit d4b220b
feat(analysis): cond5 per-idx provenance ledger (ur/zh/es)
Adds a disk-authoritative ledger of every source idx attempted across the three Cond-5 datasets, plus the extractor script. Records per (lang, split, idx) whether the row was attempted, valid, failed (InvalidCodeError), or never attempted (outside the --idx-allowlist).
summary.json is unreliable across --resume boundaries (ur undercounts), so status is derived from the on-disk .py / .error.txt artifacts. This is the machine-independent provenance record behind the paper's 'attempted N, valid M' framing; packaged/ itself is git-ignored and not on HF.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Madison (Pfaff) Edgar <7844510+madiedgar@users.noreply.github.com>1 parent e0b4011 commit d4b220b
3 files changed
Lines changed: 15266 additions & 0 deletions
File tree
- expedition-tiny-aya/analysis
- scripts
0 commit comments