feat(transcription): add voice/video-note transcription - #197
Open
PathKnower wants to merge 3 commits into
Open
Conversation
record["text"] was built directly from msg.message without falling back to get_media_label(msg), so a voice message, photo, or any other captionless media item rendered identically to an actually-empty message. message_to_dict (used by get_history) already handled this correctly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds transcribe_voice, a dual-engine transcription tool: Groq-hosted whisper-large-v3-turbo (default, leaves the server but preserves the full recording) and native Telegram Premium transcription (free, audio never leaves Telegram, but empirically drops the last speech segment in roughly 2 of 3 recordings, so it's kept as an explicit opt-in fallback rather than the default). Native results are polled while pending instead of returned as truncated text. Results are cached by (chat_id, message_id) in a dedicated SQLite file (0600/0700 permissions, mounted as its own Docker volume so it survives a rebuild) since Groq transcription is a real per-call cost, not free like native. get_history/get_messages/list_messages now surface cached transcripts inline instead of showing empty text for voice messages; TELEGRAM_TRANSCRIBE controls whether listings only show what's already cached (default) or also prefetch missing ones, bounded by a per-call budget so one large listing can't trigger dozens of downloads+uploads. Engine selection is configurable both ways: an explicit engine= argument on transcribe_voice, and a TELEGRAM_TRANSCRIBE_ENGINE default for callers that don't pass one. Every transcript is returned/rendered with an explicit note that it's a machine transcript, not a verbatim quote. transcribe_voice is read-only-annotated so it stays exposed under TELEGRAM_EXPOSED_TOOLS= read-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A cached telegram transcript was answering groq requests. The lookup ran before the engine was resolved and matched any source, so an explicit engine=groq silently returned the native text - and the native engine drops the recording's last speech segment in roughly 2 of 3 recordings, a loss that is invisible in the text itself. The cache is now keyed by (chat_id, message_id, source) and the tool resolves the engine before looking anything up. Unpinned callers (listing render, backfill skip check) still accept any source and prefer the default engine, so display and quota behaviour are unchanged. Existing caches are rebuilt in place on first use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #196 (small, unrelated
list_messagesmedia-label fix) — thisbranch is #196 plus the transcription feature on top. The diff below will
shrink to just the feature once #196 merges; happy to rebase if you'd rather
review it standalone.
Summary
transcribe_voice(chat_id, message_id, engine=None)tool: transcribes a voice message or video note. Two engines behind one interface:groq(default): uploads the recording to Groq's hostedwhisper-large-v3-turbo. Costs a download+upload per call and leaves the server, but doesn't truncate long recordings.telegram: nativemessages.TranscribeAudioRequest. Free, audio never leaves Telegram, requires Premium on the account — but in side-by-side testing on real recordings it silently dropped the last spoken clause in roughly 1 of 3 samples (confirmed: native output ended mid-sentence where Groq's output on the identical file continued for another clause). Long recordings returnpending; the tool polls automatically instead of returning truncated/empty text.engine=argument per call, orTELEGRAM_TRANSCRIBE_ENGINEas the server-wide default.(chat_id, message_id)in a dedicated SQLite file (own directory, 0600/0700 permissions) since Groq transcription is a real per-call cost, not free like native — a repeat read/listing never re-transcribes the same message.get_history,get_messages, andlist_messagesnow surface already-cached transcripts inline instead of leaving voice messages as empty text.TELEGRAM_TRANSCRIBE(off/on-demanddefault /auto) controls whether listings only show what's cached or also prefetch missing ones, bounded by a per-call budget (TELEGRAM_TRANSCRIBE_MAX_VOICES/_MAX_SECONDS) so one large listing can't silently trigger dozens of paid downloads.transcribe_voiceisreadOnlyHint=True-annotated, so it stays exposed underTELEGRAM_EXPOSED_TOOLS=read-only../transcript_cachevolume so the cache survives a rebuild (previously would've lived only in the container's writable layer).This is a bigger surface than a typical PR here — new tool, new env vars, a paid third-party dependency by default — so happy to adjust scope/defaults (e.g. default to
telegraminstead ofgroq, make the feature fully opt-in viaTELEGRAM_TRANSCRIBE=off, split cache/budget into a follow-up) if you'd rather land a smaller version first.Test plan
Deployed to a live server and walked through both engines against 11 real voice messages in a real chat (15s–159s recordings): all transcribed successfully, repeat calls served from cache (no second API call), and the native-vs-Groq truncation difference reproduced on one of the recordings as described above.