Skip to content

feat(transcription): add voice/video-note transcription - #197

Open
PathKnower wants to merge 3 commits into
chigwell:mainfrom
khairutdinov-personal:feat/voice-transcription
Open

feat(transcription): add voice/video-note transcription#197
PathKnower wants to merge 3 commits into
chigwell:mainfrom
khairutdinov-personal:feat/voice-transcription

Conversation

@PathKnower

Copy link
Copy Markdown
Contributor

Stacked on #196 (small, unrelated list_messages media-label fix) — this
branch is #196 plus the transcription feature on top. The diff below will
shrink to just the feature once #196 merges; happy to rebase if you'd rather
review it standalone.

Summary

  • New transcribe_voice(chat_id, message_id, engine=None) tool: transcribes a voice message or video note. Two engines behind one interface:
    • groq (default): uploads the recording to Groq's hosted whisper-large-v3-turbo. Costs a download+upload per call and leaves the server, but doesn't truncate long recordings.
    • telegram: native messages.TranscribeAudioRequest. Free, audio never leaves Telegram, requires Premium on the account — but in side-by-side testing on real recordings it silently dropped the last spoken clause in roughly 1 of 3 samples (confirmed: native output ended mid-sentence where Groq's output on the identical file continued for another clause). Long recordings return pending; the tool polls automatically instead of returning truncated/empty text.
  • Engine choice is configurable two ways: the engine= argument per call, or TELEGRAM_TRANSCRIBE_ENGINE as the server-wide default.
  • Results are cached by (chat_id, message_id) in a dedicated SQLite file (own directory, 0600/0700 permissions) since Groq transcription is a real per-call cost, not free like native — a repeat read/listing never re-transcribes the same message.
  • get_history, get_messages, and list_messages now surface already-cached transcripts inline instead of leaving voice messages as empty text. TELEGRAM_TRANSCRIBE (off / on-demand default / auto) controls whether listings only show what's cached or also prefetch missing ones, bounded by a per-call budget (TELEGRAM_TRANSCRIBE_MAX_VOICES / _MAX_SECONDS) so one large listing can't silently trigger dozens of paid downloads.
  • Every transcript is returned/rendered with an explicit note that it's a machine transcript, not a verbatim quote — wording drifts under both engines, so it shouldn't be treated as an exact quote of what someone said.
  • transcribe_voice is readOnlyHint=True-annotated, so it stays exposed under TELEGRAM_EXPOSED_TOOLS=read-only.
  • Docker Compose now mounts a ./transcript_cache volume so the cache survives a rebuild (previously would've lived only in the container's writable layer).

This is a bigger surface than a typical PR here — new tool, new env vars, a paid third-party dependency by default — so happy to adjust scope/defaults (e.g. default to telegram instead of groq, make the feature fully opt-in via TELEGRAM_TRANSCRIBE=off, split cache/budget into a follow-up) if you'd rather land a smaller version first.

Test plan

uv run pytest tests/ -q
# 361 passed (80 new: tests/test_transcription.py, tests/test_voice_transcripts.py)
uv run pre-commit run --all-files
black....................................................................Passed
flake8...................................................................Passed

Deployed to a live server and walked through both engines against 11 real voice messages in a real chat (15s–159s recordings): all transcribed successfully, repeat calls served from cache (no second API call), and the native-vs-Groq truncation difference reproduced on one of the recordings as described above.

PathKnower and others added 3 commits August 21, 2026 00:30
record["text"] was built directly from msg.message without falling back to
get_media_label(msg), so a voice message, photo, or any other captionless
media item rendered identically to an actually-empty message. message_to_dict
(used by get_history) already handled this correctly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds transcribe_voice, a dual-engine transcription tool: Groq-hosted
whisper-large-v3-turbo (default, leaves the server but preserves the full
recording) and native Telegram Premium transcription (free, audio never
leaves Telegram, but empirically drops the last speech segment in roughly
2 of 3 recordings, so it's kept as an explicit opt-in fallback rather than
the default). Native results are polled while pending instead of returned
as truncated text.

Results are cached by (chat_id, message_id) in a dedicated SQLite file
(0600/0700 permissions, mounted as its own Docker volume so it survives a
rebuild) since Groq transcription is a real per-call cost, not free like
native. get_history/get_messages/list_messages now surface cached
transcripts inline instead of showing empty text for voice messages;
TELEGRAM_TRANSCRIBE controls whether listings only show what's already
cached (default) or also prefetch missing ones, bounded by a per-call
budget so one large listing can't trigger dozens of downloads+uploads.

Engine selection is configurable both ways: an explicit engine= argument
on transcribe_voice, and a TELEGRAM_TRANSCRIBE_ENGINE default for callers
that don't pass one. Every transcript is returned/rendered with an explicit
note that it's a machine transcript, not a verbatim quote. transcribe_voice
is read-only-annotated so it stays exposed under TELEGRAM_EXPOSED_TOOLS=
read-only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A cached telegram transcript was answering groq requests. The lookup ran
before the engine was resolved and matched any source, so an explicit
engine=groq silently returned the native text - and the native engine
drops the recording's last speech segment in roughly 2 of 3 recordings,
a loss that is invisible in the text itself.

The cache is now keyed by (chat_id, message_id, source) and the tool
resolves the engine before looking anything up. Unpinned callers
(listing render, backfill skip check) still accept any source and
prefer the default engine, so display and quota behaviour are unchanged.
Existing caches are rebuilt in place on first use.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant