Skip to content

feat(chat): add Re-run button to the vision text editor - #5968

Open
JGW3 wants to merge 2 commits into
odysseus-dev:devfrom
JGW3:feat/vision-editor-rerun
Open

feat(chat): add Re-run button to the vision text editor#5968
JGW3 wants to merge 2 commits into
odysseus-dev:devfrom
JGW3:feat/vision-editor-rerun

Conversation

@JGW3

@JGW3 JGW3 commented Aug 9, 2026

Copy link
Copy Markdown

Summary

GET /api/upload/{file_id}/vision already accepts force=1 to bypass the cached description and re-run the vision model, but nothing in the UI ever passes it — _openVisionEditor fetches the endpoint without the parameter, so the editor can only ever show the cached text. There is currently no way to regenerate a description after switching the Vision model in Settings, or after a first run produced a poor result. Adds a Re-run button beside Close that calls the existing endpoint with force=1 and loads the result into the textarea. No new endpoint, no backend change.

Note: get_vision_text already calls _sync_gallery_caption_for_upload on both the cached and recomputed paths, so a re-run also refreshes the image's gallery caption. That is pre-existing endpoint behaviour — this PR only surfaces the force=1 path that was already implemented.

Target branch

  • This PR targets dev, not main

Linked Issue

Fixes #5983

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — this is not a duplicate.
  • This PR targets dev
  • My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
  • I actually ran the app (docker compose up or uvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.

How to Test

  1. Attach an image to a chat message and let the vision description generate.
  2. Change Settings → Vision to a different vision-capable model.
  3. Open the image's Vision text editor.
  4. Before this change: the editor shows the description from the old model with no way to regenerate it — Save only persists hand edits, and Regenerate message resends the chat turn without re-running vision.
  5. After this change: click Re-run; the textarea repopulates with a fresh description from the currently configured model.

Verified against two models in sequence (qwen3-vl:8b-instruct and a Gemma 4 build) — the re-run returns visibly different text for the same image, confirming it is hitting the model rather than the cache.

Environment tested: Docker deployment, Ollama backend, Ubuntu, Firefox.

Visual / UI changes — REQUIRED if you touched anything that renders

  • Screenshot of the change in the running app, attached below.
  • Style match: the new button reuses the existing vision-editor-btn class already used by Close in the same row — no new CSS, no new color values, font sizes, or spacing units. The icon is an inline monochrome SVG matching the stroke style of the existing editor icons. No emoji. No font overrides. No hard-coded theme values.
  • No new component patterns. The button is appended to the existing actions row alongside Close / Save / Regenerate message; nothing parallel was introduced.
  • I am not an LLM agent submitting a bulk PR. Found while using the app on a self-hosted instance; an assistant helped draft this description. The commit carries a Co-Authored-By trailer.

Screenshots / clips

Screenshot 2026-08-10 154020 Screenshot 2026-08-10 154153

@github-actions github-actions Bot added the needs work PR description incomplete — please update before review label Aug 9, 2026
The vision-text endpoint already accepts force=1 to recompute an
image description, but the editor only ever fetches the cached value —
there is no way to regenerate a caption after switching Vision models.
Adds a Re-run button beside Close that calls the existing endpoint with
force=1 and loads the result into the textarea.

Co-Authored-By: Claude <noreply@anthropic.com>
@JGW3
JGW3 force-pushed the feat/vision-editor-rerun branch from a7cc1e7 to e3c72e9 Compare August 10, 2026 21:04
@github-actions github-actions Bot added ready for review Description complete — ready for maintainer review and removed needs work PR description incomplete — please update before review labels Aug 10, 2026
@JGW3
JGW3 marked this pull request as ready for review August 10, 2026 21:48

@RaresKeY RaresKeY left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the latest head and found one issue in the regenerate path.

Findings

P2 Badge issue (chat): Regenerate message now appends instead of replacing from the edited turn

  • Problem: This change removes { replaceFromHere: true } from the existing vision-editor regenerate call. resendUserMessage defaults that option to false, so “Regenerate message” takes the ordinary non-destructive resend path.

  • Impact: Saving edited vision text and regenerating keeps the later conversation and appends another response instead of regenerating from that turn. The current pytest run fails the existing caller-contract test for this behavior.

  • Ask: Restore window.chatModule.resendUserMessage(userMsgEl, { replaceFromHere: true });; the new Re-run action does not require changing this call.

  • Location: static/js/chatRenderer.js:367

Validation

  • Current CI reproduces this exactly: test_only_regenerate_callers_opt_into_replace_from_here fails with 1 failed, 4897 passed, 4 skipped; JavaScript syntax and the other current checks pass.

Re-run button change accidentally dropped { replaceFromHere: true }
from the existing regenerate-message resend call, so saving edited
vision text and regenerating appended instead of truncating from that
turn. Restores the flag; addresses RaresKeY's review on PR odysseus-dev#5968.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B7CCswF8829EVJuHWqjpC2
@JGW3

JGW3 commented Aug 11, 2026

Copy link
Copy Markdown
Author

That was accidental scope creep from wiring up the new Re-run button, not intentional. Restored { replaceFromHere: true } in 3cf80e8. Both test_resend_message_does_not_truncate_by_default and test_only_regenerate_callers_opt_into_replace_from_here pass locally.

@JGW3
JGW3 requested a review from RaresKeY August 12, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(gallery): add option to regenerate vision captions in chat and gallery

2 participants