Fix/surface oss-fuzz:42535468's missing libcrypto.so.1.1 (fixes #41)

[alanzabihi]

Summary

Investigated all three options in #41 for oss-fuzz:42535468 (OpenSC)'s missing libcrypto.so.1.1. Options 1 and 2 are not actionable from benchpress's own scope (details below, backed by live verification) — option 3 is implemented.

Investigation findings

Option 1 (install a libssl1.1 compat package into the running container at standUpTarget() time): not feasible. standUpTarget() only docker pulls the -vul/-fix images as artifacts — it never keeps a container running. The two places this adapter does execute anything against an image are (a) the manual verify-cybergym-standup.ts smoke test and (b) resolveFixCommitSha()'s throwaway, never-started container (docker create/docker cp/docker rm, no execution). There is no long-lived target container in this repo's control for a contender's build+run step to reuse or patch. More fundamentally: AutoBrin's reproduction is source-first by design (.agents/skills/autobrin-contributor-poc/SKILL.md in autobrin-flue: repro.sh "builds the minimal set of target sources... into its own working directory... then runs the resulting binary" — it never touches the pulled Docker image at all). The dynamic-linker failure happens in whatever sandbox runs repro.sh (autobrin-flue's own reproduction pipeline, entirely outside this adapter's provisioning), not in the pulled image.

Option 2 (a differently-tagged upstream image): not applicable. I live-verified cybergym/oss-fuzz:42535468-vul is not broken. docker run cybergym/oss-fuzz:42535468-vul /out/fuzz_pkcs15init -help=1 executes cleanly (exit 0, full libFuzzer help output); libcrypto.so.1.1 is present and correctly resolved (ldd/dpkg -S both confirm libssl1.1:amd64 is installed in the image). I also ran this repo's own scripts/verify-cybergym-standup.ts oss-fuzz:42535468 end-to-end for real: it pulls both images, extracts the real HF source, and replays the reference PoC — vulCrashed=true fixSilent=true crashTypeMatch=true (vul exit 1 / AddressSanitizer heap-buffer-overflow, fix exit 0). The dockerized build envs this adapter owns are completely healthy; a different image tag would change nothing, since these images were never the execution surface for the actual failure.

Root cause (confirmed against real production evidence in superagent-ai/autobrin-flue#190): AutoBrin's contributor builds+runs OpenSC's fuzz harness from the plain extracted source (workspace/target/src-vul/opensc/... — src-vul is literally the HF tarball's own top-level folder name, confirmed live by the standup script's own output) in whatever sandbox is running the engagement. That sandbox lacks libcrypto.so.1.1, so every attempt fails identically at the dynamic-linker stage. The evaluator stage already correctly rejects these attempts (per #190's own account), which means findConfirmedAttempts() in score.ts finds zero confirmed attempts — and today, that unconditionally scores as false_negative, indistinguishable from a genuine miss.

Option 3 (detect and surface distinctly): implemented. findEnvironmentBrokenAttempts() in score.ts reads the raw stderr_tail text of every attempt's self-test.json (written by autobrin-flue's runSelfTest()) for the glibc dynamic linker's own fixed error text (error while loading shared libraries: ... cannot open shared object file) — never the self-test's own status/passed verdict, so this is correct independent of whether #190's self-test classifier fix has landed. It only fires when every attempt that reached the self-test stage shows this exact signature (conservative: a real, unrelated miss sitting alongside a flaky environment failure is never misclassified). When it fires, scoreCyberGymClaim() returns an excluded signal tagged environmentBroken: true with the affected attempt names and a link back to this issue, instead of false_negative.

Change

• src/benchmarks/cybergym/score.ts: new findEnvironmentBrokenAttempts(), wired into scoreCyberGymClaim()'s zero-confirmed-attempts branch. Small refactor (listAttemptDirs()) shared with the existing findConfirmedAttempts(), no behavior change there.
• src/benchmarks/cybergym/README.md: documents the new "Broken target environments" policy and why options 1/2 don't apply.
• tests/cybergym-score.test.ts: 9 new tests, including one using the exact real stderr_tail text from #190's production evidence, plus a mixed-evidence case proving a genuine miss still scores as false_negative.

Live verification

Pulled cybergym/oss-fuzz:42535468-vul/-fix for real and ran both manual docker run checks and this repo's own scripts/verify-cybergym-standup.ts oss-fuzz:42535468 end-to-end (real HF download + Docker pulls + differential PoC replay) — full pass, images healthy. Docker images and .cache/ removed afterward; no leftover containers.

Test plan

• npm run validate (typecheck + full suite) green — 281/281 tests passing (9 new)
• Local Bugbot review — no findings
• Live Docker verification against the real oss-fuzz:42535468 images (see above)

[cursor]

Current version of PR was reviewed by /review-bugbot on Jul 2, 14:49 GMT+2. It flagged 0 findings.

^{Bugbot on commit b587c06 is skipped.}