Skip to content

Add Parquet/DuckDB export layer for market-inefficiency-hunter integration #16

@Sszmentor

Description

@Sszmentor

Goal

Bridge polymarket-mcp-server with market-inefficiency-hunter project. Currently zero integration surface (no polars, DuckDB, Parquet).

Requirements

1. Orderbook Snapshot Exporter

  • New module: src/polymarket_mcp/export/parquet_writer.py
  • Periodically call get_orderbook and write to Parquet
  • Schema: (timestamp: datetime, market_id: str, token_id: str, side: str, price: f64, size: f64)
  • Use polars for DataFrame construction

2. WebSocket → DuckDB Writer

  • Attach callback to WebSocketManager.notification_callback
  • Stream price_change and agg_orderbook events to DuckDB
  • DuckDB can append to Parquet efficiently

3. Optional Dependencies

  • Add [analytics] extras group in pyproject.toml: polars>=0.20, duckdb>=0.10, pyarrow>=15

4. New MCP Tools

  • export_orderbook_snapshot(market_id, output_path) — one-shot Parquet export
  • query_orderbook_history(market_id, start_ts, end_ts) — query DuckDB store

Context

  • market-inefficiency-hunter uses polars (NOT pandas), DuckDB, Parquet everywhere
  • See ~/Projects/AI_Agents/market-inefficiency-hunter/docs/decisions.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions