Skip to content

Add MCP Server for Video Navigation with Keyword Search and Visual Search#48

Draft
LemurPwned with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-aaabfe37-a59b-4418-81e6-1f5a8f253fb2
Draft

Add MCP Server for Video Navigation with Keyword Search and Visual Search#48
LemurPwned with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-aaabfe37-a59b-4418-81e6-1f5a8f253fb2

Conversation

Copilot AI commented Aug 20, 2025

Copy link
Copy Markdown

This PR implements a comprehensive Model Context Protocol (MCP) Server for video navigation that provides advanced search capabilities and intelligent video sampling job generation.

🎯 Features Implemented

Resources

  • Video Resource (video://video_id) - Access individual video metadata, subtitles, and content
  • ImageSet Resource (imageset://all_frames) - Collection of all sampled video frames across loaded videos

Tools

Search Tool - Multi-strategy video content search:

  • Keyword search in subtitles using existing KeywordExtractor with spacy-based NLP, gracefully falling back to simple text matching when spacy is unavailable
  • Visual search foundation using keyframe sampling (ready for CLIP integration)
  • YouTube URL support with automatic signed URL generation via YTDLPPlugin
  • Time range filtering with start/stop parameters for precise navigation
  • Multi-video search when no specific video path is provided

Video Sampling Job Tool - Intelligent code generation:

  • Auto-proposes Python code using existing SamplerConfig and Worker classes
  • Auto-proposes CLI commands for all supported sampling methods (hash, entropy, gzip, buffer, grid)
  • Interactive mode asks for user preference when output format isn't specified
  • Full parameter customization (frame intervals, hash sizes, buffer sizes, keyframes-only mode)

🏗️ Implementation Details

The MCP server is built on top of existing video-sampler components:

  • Leverages video_sampler.language.keyword_capture for subtitle processing
  • Uses video_sampler.integrations.yt_dlp_plugin for YouTube support
  • Integrates with video_sampler.samplers for frame extraction
  • Maintains backward compatibility with all existing functionality

📦 Usage

Start the MCP server:

video_sampler_mcp
# or 
python -m video_sampler.mcp_server.server

Search for content:

{
  "name": "search",
  "arguments": {
    "query": "wildlife documentary",
    "video_path": "https://youtube.com/watch?v=abc123",
    "start": 30.0,
    "stop": 120.0
  }
}

Generate sampling code:

{
  "name": "video_sampling_job", 
  "arguments": {
    "video_path": "/path/to/video.mp4",
    "sampling_method": "hash",
    "output_format": "python",
    "frame_interval": 2.0
  }
}

✅ Testing

  • 14/14 tests passing with comprehensive coverage
  • Tests include YouTube URL detection, code generation, video loading, subtitle search with time filtering, and error handling
  • Includes demonstration script (demo_mcp.py) showcasing all features
  • Graceful handling of missing optional dependencies (spacy, yt-dlp, openai)

📚 Documentation

  • Complete README in video_sampler/mcp_server/README.md with usage examples
  • Inline documentation for all MCP resources and tools
  • Integration examples for AI assistants and MCP clients

This implementation provides a standardized, production-ready interface for video content analysis and processing, enabling AI assistants to intelligently navigate and process video content using the full power of the video-sampler framework.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits August 20, 2025 10:57
…mpling tools

Co-authored-by: LemurPwned <8441184+LemurPwned@users.noreply.github.com>
Co-authored-by: LemurPwned <8441184+LemurPwned@users.noreply.github.com>
Copilot AI changed the title [WIP] Create an MCP server for video navigation with keyword search and visual search. Implement the folloowing: 1. Resources: - Video - ImageSet 2. Tools - Search (args: query: str, video_path: str | None, start: float|None, stop: float|None) if video p... Add MCP Server for Video Navigation with Keyword Search and Visual Search Aug 20, 2025
Copilot AI requested a review from LemurPwned August 20, 2025 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants