Skip to content

Clarify missing benchmark timing suite - #812

Merged
morluto merged 1 commit into
mainfrom
agent/diagnose-benchmark-suite-selection
Aug 8, 2026
Merged

Clarify missing benchmark timing suite#812
morluto merged 1 commit into
mainfrom
agent/diagnose-benchmark-suite-selection

Conversation

@morluto

@morluto morluto commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Fixes #722.

The timing-weight validation reads the live Harbor registry and previously selected a suite with a bare next(). If dataset changes removed every suite with more than twelve tasks, the test failed with an opaque StopIteration.

The test now asserts the expected suite explicitly and includes the observed suite task counts in the failure message. Fixture-controlled next() calls are intentionally unchanged.

Validation:

  • make harbor-validation-tests TESTS=benchmarks/validation/test_benchmark_planner.py
  • make lint-full

@morluto
morluto marked this pull request as ready for review August 8, 2026 07:06
@cursor

cursor Bot commented Aug 8, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@morluto
morluto merged commit fdc52ff into main Aug 8, 2026
32 checks passed
@morluto
morluto deleted the agent/diagnose-benchmark-suite-selection branch August 8, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Potential StopIteration in benchmark validation tests from bare next() calls

1 participant