Skip to content

feat: view peer photos through MCP — index, open and contact sheet - #199

Merged
chigwell merged 8 commits into
chigwell:mainfrom
dagahan:feature/peer-photos
Aug 23, 2026
Merged

feat: view peer photos through MCP — index, open and contact sheet#199
chigwell merged 8 commits into
chigwell:mainfrom
dagahan:feature/peer-photos

Conversation

@dagahan

@dagahan dagahan commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Photos are the one thing this server can detect but never show. get_media_label correctly classifies an attached photo, get_user_photos returns bare integers, and get_chat reports has_photo: true — but the only route to the actual bytes is download_media, which writes to disk and returns a path. An MCP client without filesystem access simply cannot see a Telegram photo.

This adds three tools that close that gap for any peer — user, basic group, supergroup or channel — and surfaces the profile fields GetFullUser already returns.

Design: a tool either returns text or returns images, never conditionally

Vision tokens are expensive, so cost is predictable from the tool name alone rather than hidden behind a parameter:

Tool Cost Returns
list_photos free JSON index of ids, dates, captions
open_photo 1 image one photo, full resolution
get_photo_sheet 1 image labelled collage of the last N

Each cell of the sheet is stamped with the exact id open_photo accepts, so the collage is a navigable index rather than a preview. Discovery stays free; you pay only for what you decide to look at.

list_photos(peer)      → text index, no image transferred
get_photo_sheet(peer)  → six avatars for one image of tokens, each cell labelled
open_photo(peer, id)   → that one, full resolution

Three sources, one resolver

telegram_mcp/photo_source.py is the only place that branches on peer type:

source User Group / supergroup / channel
avatars GetUserPhotosRequest InputMessagesFilterChatPhotos (no equivalent API exists)
messages InputMessagesFilterPhotos InputMessagesFilterPhotos

All three converge on one PhotoReference, so the tools never branch and the sheet builder exists once. Bytes go straight to memory via download_media(..., file=bytes); the sheet requests Telegram's pre-rendered small sizes rather than full images.

Two pre-existing bugs this arms, fixed first

Both are independent of the feature and reachable by any tool returning non-text:

  • with_account corrupts non-string returns. The multi-account read-only fan-out joins results with "\n\n".join(f"[{label}]\n{result}"), which stringifies an Image into its repr. It now emits content blocks when any account yields non-text, and keeps the joined string when all yield text.
  • The audience=["user"] annotation hook skips images. _install_annotation_hook only tagged TextContent, so image blocks would reach clients without the user-generated-data marker that the hook exists to apply. Now covers ImageContent.

Profile fields

get_full_user discarded most of what GetFullUser returns. Added, all additive and sanitized: collectible usernames, language, business profile (location/hours/intro), trust flags (scam/fake/restricted + reasons), relationship flags, and current_avatar_id. get_chat gains the same avatar id for every peer type. The id comes off the entity, so none of this costs an extra round trip.

Conformance

  • readOnlyHint=True on all three tools, so TELEGRAM_EXPOSED_TOOLS=read-only keeps them
  • @validate_id, @with_account(readonly=True), log_and_format_error, sanitize_user_content throughout
  • MCP resources were considered and rejected: resources/read bypasses the annotation hook, the TELEGRAM_EXPOSED_TOOLS gate and with_account, so a resource surface would silently disable three safeguards and break under multi-account
  • Degrades rather than fails — Pillow absent disables only the sheet, an unsearchable channel returns an empty index, a single avatar returns no sheet

Dependency

Pillow, in both pyproject.toml and requirements.txt since the Dockerfile installs from the latter. ffmpeg's tile filter was the alternative but common builds omit drawtext, which would lose the cell labels that make the sheet useful. Pillow ships cp313 musllinux wheels, so the Alpine image needs no extra system packages — verified by building it.

Testing

50 new tests; 335 pass overall, up from 281. black and flake8 clean, Docker image builds.

Verified against the live API, not only fixtures: avatar and message sheets, full-resolution opens addressed from sheet labels in both directions, the chat-photo service-message branch on a real supergroup, and the error paths for unknown ids and invalid sources.

Live testing also caught something fixtures could not — GetUserPhotos returns avatars in profile display order, not chronologically. Since that order is the peer's own arrangement, sorting would discard real information; the docstrings now state it instead.

🤖 Generated with Claude Code

dagahan and others added 8 commits August 22, 2026 22:42
Declared in both pyproject.toml and requirements.txt because the Dockerfile
installs from requirements.txt; the two must not drift.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The multi-account read-only fan-out joined results with str formatting, which
stringifies an Image into its repr. It now returns content blocks when any
account yields non-text, and keeps the joined string when all yield text.

The audience=["user"] annotation hook only tagged TextContent, so image blocks
would reach clients without the user-generated-data marker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lays out (bytes, label) pairs row-major into one JPEG. Each cell carries the
identifier used to open that image at full resolution, so the sheet works as
an index rather than a preview. Pillow is imported lazily so its absence
degrades to a typed error instead of breaking import of the tool modules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Users expose avatar history through GetUserPhotos; groups and channels have
no equivalent, so their past avatars are recovered from the chat-photo service
messages. Message photos come from the photo message filter and are addressed
by message id. All three converge on one PhotoReference so callers never
branch on peer type.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
list_photos indexes any peer's avatars or posted photos as text alone, so
discovery costs no vision tokens. open_photo and get_photo_sheet are the two
tools that return image content, making the cost of a call predictable from
its name. Sheet cells are labelled with the id open_photo accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GetFullUser carried collectible usernames, language, business profile, trust
and relationship flags, and the current avatar id; all were discarded. The
avatar id comes off the entity, so nothing here costs an extra round trip.
get_chat gains the same id for groups, supergroups and channels.

Additive only: every previously returned key keeps its name and meaning.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ical

GetUserPhotos returns avatars in the order the peer arranged them on their
profile, which live testing confirmed is not date-descending. Sorting would
discard that choice, so the ordering stands and the docstrings now say what it
actually is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fixtures carried a real Telegram handle and numeric user id, which must
not be published in a public repository.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chigwell
chigwell merged commit 52cca20 into chigwell:main Aug 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants