Skip to content

Commit e0d3f5d

Browse files
authored
Document Inspect AI version troubleshooting (#10)
Thanks @adminlip for the troubleshooting documentation. Merging — content matches what issue #9 asked for. Closing #9 as resolved by this PR.
1 parent f6d0749 commit e0d3f5d

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,16 @@ Exit codes:
9292
- `3` — tamper (hash mismatch — fields changed after pre-registration)
9393
- `2` — log not found / structurally invalid
9494

95+
## Inspect AI version troubleshooting
96+
97+
`falsify-inspect` 0.1.x supports the Inspect AI eval log shape produced by `inspect_ai>=0.3.0`, which is the version range installed by the optional `inspect` extra. If `falsify-inspect verify` reports that a log is structurally invalid, cannot find the expected score/metadata fields, or raises a parsing error immediately after an Inspect AI upgrade, first confirm that the package versions are in sync:
98+
99+
```bash
100+
python -m pip show falsify-inspect inspect_ai
101+
```
102+
103+
When the log was generated with a newer Inspect AI release, retry verification in an environment using the supported range, or regenerate the log after upgrading `falsify-inspect` to a release that documents support for the newer Inspect AI schema. If the versions look compatible, keep the failing `eval.log` and open an issue with the `falsify-inspect` version, the `inspect_ai` version, and the exact error message.
104+
95105
## What this plugin does *not* do
96106

97107
- Does not modify `inspect_ai` itself. It reads existing eval log JSON.

0 commit comments

Comments
 (0)