chore(harbor): pin the default fork to 0ab5968 for the gemini flash fix - #1029
Draft
RishiDesai wants to merge 1 commit into
Draft
chore(harbor): pin the default fork to 0ab5968 for the gemini flash fix#1029RishiDesai wants to merge 1 commit into
RishiDesai wants to merge 1 commit into
Conversation
Picks up Harbor PR #16. Before it, Gemini CLI's legacy model resolution rewrote any model id ending in "flash" to whichever flash model the installed CLI considered current, so a trial submitted as google/gemini-3.6-flash silently ran on gemini-3.5-flash and recorded the wrong model in its trajectory. The range from the previous pin touches only the gemini-cli agent and its tests, so nothing else in the worker image changes. Verified on this commit before the merge: trial hello-world-b604edb0-124 (harbor_sha 1dbf3e4e, reasoning_effort=high) recorded gemini-3.6-flash in the CLI's own session trajectory and in ATIF model_name, with no occurrence of gemini-3.5-flash anywhere in the trial. Co-authored-by: rishi <rishi@abundant.ai>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Oddish previewCommit:
Vercel deployment URL: https://oddish-6lf8a1vcl.oddish.app Plan:
This comment is updated by the PR Preview workflow. |
dot-agi
force-pushed
the
staging
branch
2 times, most recently
from
August 4, 2026 04:48
bb5d394 to
9e715c4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the default Harbor pin from
f6b86c4to0ab5968to pick up Harbor PR #16.Why
Before that fix, Gemini CLI's legacy model resolution treated any model id ending in
flashas a request for "the current flash model" and rewrote it to whatever the installed CLI considered current:So a trial submitted as
google/gemini-3.6-flashsilently ran ongemini-3.5-flashand recorded the wrong model in its trajectory — no error, no warning. Any Oddish trial on a*flashmodel id newer than the pinned CLI's built-in list was affected, with or withoutreasoning_effort.Changes
The usual three files for a re-pin:
oddish/src/oddish/config.py—HARBOR_DEFAULT_SHA, whichSettings.harbor_source_refderives from, so the probe ref moves with the worker image by constructionoddish/uv.lock,backend/uv.lock— the resolved commit for themain-branch git sourceThe range from the previous pin (
f6b86c4..0ab5968) touches onlysrc/harbor/agents/installed/gemini_cli.pyand its unit tests, so nothing else in the worker image changes.Verification
The merge commit's tree is byte-identical to the fix commit that was smoke-tested (
git diff 1dbf3e4e 0ab5968fis empty; both are tree58020900), so the trial evidence applies exactly to this pin.Oddish trial
hello-world-b604edb0-124(experiment089dcded), submitted asgoogle/gemini-3.6-flashwithreasoning_effort=highonmodaland pinned toharbor_sha 1dbf3e4e, finishedsuccesswith reward 1.0. From the Gemini CLI's own session trajectory — its record of the model it actually called, not Oddish's submitted label:ATIF
trajectory.jsonagrees atagent.model_nameand per-step. Across all artifacts pulled for that trial: 28 occurrences ofgemini-3.6-flash, 12 of the Harbor aliasgemini-3.6-flash-high, and zero ofgemini-3.5-flash.After the re-pin, the installed Harbor carries both parts of the fix (
chat-base-3inheritance anddynamicModelConfiguration), confirmed by inspectingGeminiCli._build_settings_configin the synced venv.Tests
The pin-drift guards pass, including
test_probe_harbor_ref_matches_pyproject_pinand thebackend/uv.lockdrift assertion intest_harbor_spec_parse.py(42 passed acrosstest_harbor_*andtest_probe_creds_injection.py). Both lockfiles validate underuv sync --frozen, which is what CI uses.The full
oddish/suite reports the same 327 failures before and after this change — a byte-identical list, all from this sandbox having no Postgres. This re-pin introduces no new failures.I reverted one piece of incidental collateral: re-locking with a newer
uvalso dropped thesys_platformmarkers fromsecretstorage's dependencies in both lockfiles. That is unrelated to the Harbor pin and would have changed install behavior on win32/emscripten, so the diff is now just the Harbor SHA in each lockfile.