Skip to content

fix: update Google/Vertex connection test model to gemini-2.5-flash - #996

Closed
pelazas wants to merge 1 commit into
lfnovo:mainfrom
pelazas:fix/google-connection-test-model
Closed

fix: update Google/Vertex connection test model to gemini-2.5-flash#996
pelazas wants to merge 1 commit into
lfnovo:mainfrom
pelazas:fix/google-connection-test-model

Conversation

@pelazas

@pelazas pelazas commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Description

Testing a perfectly valid Google AI key on the Manage Models page fails with a 404, because the connection test calls gemini-2.0-flash — which Google has retired. Same for the Vertex entry.

This swaps both TEST_MODELS entries to gemini-2.5-flash, as suggested in the issue thread. Also updated the stale TEST_MODELS snippet in open_notebook/ai/CLAUDE.md so the docs match the code.

Left alone on purpose: the static VERTEX_MODELS discovery list in api/credentials_service.py also carries some older model names, but that's model discovery, not the connection test — happy to do a follow-up if you want it refreshed too.

Related Issue

Fixes #970

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring (no functional changes)
  • Performance improvement
  • Test coverage improvement

How Has This Been Tested?

  • Tested locally with Docker
  • Tested locally with development setup
  • Added new unit tests
  • Existing tests pass (uv run pytest)
  • Manual testing performed (describe below)

Test Details:

Added a regression test in tests/test_credentials_api.py pinning the google/vertex test models to gemini-2.5-flash (it fails on main, passes here). Full suite: 223 passed. ruff check clean. I didn't run a live key test against Google's API, but the change is exactly the model swap suggested by @lfnovo in the issue.

Design Alignment

Which design principles does this PR support? (See DESIGN_PRINCIPLES.md)

  • Privacy First
  • Simplicity Over Features
  • API-First Architecture
  • Multi-Provider Flexibility
  • Extensibility Through Standards
  • Async-First for Performance

Explanation:

Keeps the Google/Vertex provider path actually usable — a key test that always fails defeats the point of supporting the provider.

Checklist

Code Quality

  • My code follows PEP 8 style guidelines (Python)
  • My code follows TypeScript best practices (Frontend)
  • I have added type hints to my code (Python)
  • I have added JSDoc comments where appropriate (TypeScript)
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings or errors

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I ran linting: make ruff or ruff check . --fix
  • I ran type checking: make lint or uv run python -m mypy . (mypy isn't in the dev deps of my env; change is a string-literal swap)

Documentation

  • I have updated the relevant documentation in /docs (if applicable) — updated the TEST_MODELS snippet in open_notebook/ai/CLAUDE.md
  • I have added/updated docstrings for new/modified functions
  • I have updated the API documentation (if API changes were made)
  • I have added comments to complex logic

Database Changes

  • I have created migration scripts for any database schema changes (in /migrations)
  • Migration includes both up and down scripts
  • Migration has been tested locally

Breaking Changes

  • This PR includes breaking changes
  • I have documented the migration path for users
  • I have updated MIGRATION.md (if applicable)

Screenshots (if applicable)

N/A

Additional Context

The issue is labeled ready + good first issue and had been quiet for a bit, so I picked it up — hope that's fine. Happy to adjust if you'd rather use a different model name.

Pre-Submission Verification

  • I have read CONTRIBUTING.md
  • I have read DESIGN_PRINCIPLES.md
  • This PR addresses an approved issue that was assigned to me (approved/ready, but not formally assigned — see Additional Context)
  • I have not included unrelated changes in this PR
  • My PR title follows conventional commits format (e.g., "feat: add user authentication")

Review in cubic

The connection test hardcoded gemini-2.0-flash, which Google has
retired, so testing a valid Google AI key failed with 404 NOT_FOUND.

Fixes lfnovo#970

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

lfnovo added a commit to Thiamath/open-notebook that referenced this pull request Jul 10, 2026
Several IDs the PR introduced don't exist or are near shutdown, verified
against Google's official model/deprecation pages (2026-07):
- connection test model gemini-2.5-flash -> gemini-3.5-flash (2.5-flash
  retires 2026-10-16; 3.5-flash is the current stable GA and Google's
  named replacement, so the lfnovo#970 fix doesn't re-break in ~3 months)
- gemini-3.5-pro (does not exist) -> gemini-2.5-pro in docs
- plain gemini-3.1-flash (not a GA Gemini-API id) dropped from the Vertex
  discovery list / preferences / docs; use gemini-3.5-flash or -flash-lite
- dead gemini-pro dropped from preferences and docs
Ported lfnovo#996's lfnovo#970 regression test, pinned to gemini-3.5-flash.
@lfnovo lfnovo mentioned this pull request Jul 10, 2026
44 tasks
lfnovo added a commit that referenced this pull request Jul 10, 2026
* fix: update Google model version in connection tester and tests

* fix: updated the gemini model lists that references deprecated models

* fix: update also documentation. This is more prone to not follow the maintainer's directives, so PTADL

* fix: forgot two references to gemini deprecated versions

* fix: use valid, longer-lived gemini model IDs

Several IDs the PR introduced don't exist or are near shutdown, verified
against Google's official model/deprecation pages (2026-07):
- connection test model gemini-2.5-flash -> gemini-3.5-flash (2.5-flash
  retires 2026-10-16; 3.5-flash is the current stable GA and Google's
  named replacement, so the #970 fix doesn't re-break in ~3 months)
- gemini-3.5-pro (does not exist) -> gemini-2.5-pro in docs
- plain gemini-3.1-flash (not a GA Gemini-API id) dropped from the Vertex
  discovery list / preferences / docs; use gemini-3.5-flash or -flash-lite
- dead gemini-pro dropped from preferences and docs
Ported #996's #970 regression test, pinned to gemini-3.5-flash.

---------

Co-authored-by: Luis Novo <lfnovo@gmail.com>
@lfnovo

lfnovo commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Closing as a duplicate — the broader cleanup in #1027 (just merged) subsumes this, and it credits your PR. Thank you for catching the root issue (#970) first and for the regression test; I ported exactly that test into #1027, pinned to gemini-3.5-flash rather than gemini-2.5-flash. Heads-up for context: gemini-2.5-flash (which this PR used) is already scheduled for shutdown on 2026-10-16, so it would have re-broken the connection test in ~3 months — gemini-3.5-flash is the current stable GA and Google's named replacement. Appreciate the fix!

@lfnovo

lfnovo commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Follow-up for context: #1035 addresses the root cause you first flagged in #970. Rather than tracking Google's model ids (each has a hard-shutdown date, so it's a treadmill), it switches the Google/Vertex test to Google's floating gemini-flash-latest alias and reframes the connection test so a retired model reports the key as valid, not the connection as broken. Your regression test lives on (updated to pin the alias). Thanks for surfacing this first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Google AI connection test checks for removed gemini-2.0-flash

2 participants