Expand Tiingo REST and WebSocket API coverage

.github/workflows/ci.yml

@@ -73,7 +73,7 @@ jobs:
 
       - name: Audit dependencies, licenses, and sources
         if: matrix.rust == 'stable'
-        run: cargo deny check
+        run: cargo deny check all
 
   native-artifact:
     name: Native artifact (${{ matrix.target }})

AGENTS.md

@@ -1,14 +1,29 @@
 # AGENTS.md
 
-This is the canonical repository guide for every coding agent and harness working on `tiingo-mcp`. `CLAUDE.md` is a symlink to this file so the guidance stays identical across tools.
+This is the canonical repository guide for every coding agent and harness working on `tiingo-mcp`. `CLAUDE.md` is a symlink to this file.
+
+## Project map
+
+Following [OpenAI Harness Engineering](https://openai.com/index/harness-engineering/), this file is a concise map; use the focused references for durable detail.
+
+- [ARCHITECTURE.md](ARCHITECTURE.md) — module boundaries, REST safety, upstream WebSocket lifecycle, and process ownership.
+- [API_SURFACE.md](API_SURFACE.md) — exact tool-to-route/access/test matrix and excluded surfaces.
+- [QUALITY.md](QUALITY.md) — local/CI gates, coverage, deterministic/live evidence, and documentation contracts.
+- [README.md](README.md) — installation, MCP configuration, tools, entitlements, resources, and prompts.
+- [CHANGELOG.md](CHANGELOG.md) — released and unreleased user-facing changes.
+
+## Public contract
+
+- Expose 38 tools while preserving the original 17 tool names, required inputs, omission behavior, and results; four approved optional `columns` additions are the only legacy descriptor deltas.
+- Preserve three fixed resources, one resource template, five prompts, stdio transport, JSON text compatibility, structured content, and `TIINGO_API_KEY` authentication.
+- Tiingo WebSockets are upstream data connections exposed through four finite tools. Do not describe or implement an MCP WebSocket or Streamable HTTP transport.
 
 ## Operating rules
 
 - Keep the implementation Rust-only. Do not reintroduce Python packaging, virtual environments, or `uvx` compatibility.
-- Preserve the public MCP contract unless the task explicitly changes it: 17 tools, three fixed resources, one resource template, five prompts, stdio transport, and `TIINGO_API_KEY` authentication.
 - Keep stdout protocol-only. Send diagnostics to stderr and never log credentials or authorization headers.
 - Make surgical changes and preserve unrelated user work. Inspect `git status` before editing.
-- Do not run the quota-consuming live test without explicit authorization.
+- Do not run ignored, quota-consuming live tests without explicit authorization.
 - Do not publish crates, create or push tags, create releases, or upload MCPB bundles without explicit authorization.
 
 ## Commands
@@ -17,71 +32,26 @@ This is the canonical repository guide for every coding agent and harness workin
 cargo fmt --check
 cargo clippy --all-targets --all-features -- -D warnings
 cargo test --all-targets --locked
+cargo llvm-cov --all-targets --all-features --locked --fail-under-lines 93 --summary-only
 cargo build --release --locked
-cargo deny check
+cargo deny check all
 cargo run --quiet -- --version
-TIINGO_API_KEY=key cargo run --quiet
 dist plan
 ```
 
-The test suite binds local mock HTTP servers. The ignored live-smoke test consumes quota and must be invoked deliberately with `TIINGO_API_KEY`.
+The normal suite binds local mock servers and is credential-free. Ignored live tests consume quota or bandwidth and require explicit authorization plus `TIINGO_API_KEY`.
 
 To build an MCPB bundle from an existing target binary directory:
 
 ```bash
 bash packaging/mcpb/package.sh <target-triple> <binary-directory>
 ```
 
-## Architecture
-
-This is a native RMCP stdio server wrapping the Tiingo financial-data REST API.
-
-- `src/main.rs` parses the CLI, initializes stderr-only tracing, and starts the RMCP stdio service. `--help` and `--version` exit without starting MCP.
-- `src/lib.rs` exports the server modules used by the binary and integration tests.
-- `src/config.rs` loads `TIINGO_API_KEY` lazily so discovery, resources, and prompts work without credentials.
-- `src/client/mod.rs` owns the shared `reqwest::Client`, request bounds, retry policy, same-origin enforcement, and family-specific clients.
-- `src/client/{eod,iex,forex,crypto,news,fundamentals,corporate_actions}.rs` maps tool inputs to exact Tiingo routes and query parameters.
-- `src/client/query.rs` serializes optional query values without sending absent fields.
-- `src/error.rs` classifies configuration, validation, transport, HTTP, and response-size failures and maps them to sanitized MCP tool errors.
-- `src/mcp/tools.rs` exposes the 17 typed tools and returns both legacy JSON text and structured content.
-- `src/mcp/resources.rs` serves three fixed resources plus the `tiingo://guide/{asset_class}` template from embedded JSON under `src/mcp/data/`.
-- `src/mcp/prompts.rs` exposes the five compatible, corrected analysis prompts.
-
-## Runtime contracts
-
-- `tiingo-mcp` uses stdio transport by default. Protocol output belongs on stdout; diagnostics belong on stderr.
-- Closing stdin must terminate the process promptly and without stdout noise.
-- The credential contract is `TIINGO_API_KEY`. Never log it or the authorization header.
-- Requests must remain on the configured Tiingo origin, retry at most three total attempts, and enforce the 8 MiB decoded-response limit.
-- Recoverable tool failures set MCP `isError: true` while retaining the sanitized JSON text block expected by older clients.
-- Optional MCP tool inputs use closed JSON schemas and omit absent Tiingo query parameters.
-
-## Tests
-
-- `tests/client_http.rs` covers authentication, validation, retries, redaction, origin safety, and response bounds.
-- `tests/client_market_routes.rs` and `tests/client_data_routes.rs` cover exact REST paths and query mappings.
-- `tests/mcp_contract.rs` compares discovery against `tests/contract/baseline/v1-mcp.json`, the frozen v1 parity oracle, plus only the approved correctness deltas.
-- `tests/mcp_tools.rs`, `tests/mcp_resources.rs`, and `tests/mcp_prompts.rs` cover the RMCP surface.
-- `tests/stdio_process.rs` covers negotiation, stdout purity, CLI behavior, and EOF shutdown in a child process.
-- `tests/live_smoke.rs` keeps offline validators enabled and the quota-consuming live test ignored by default.
-
-## Distribution
-
-`cargo-dist` produces shell and PowerShell installers and native artifacts for:
-
-- `aarch64-apple-darwin`
-- `x86_64-apple-darwin`
-- `aarch64-unknown-linux-musl`
-- `x86_64-unknown-linux-musl`
-- `x86_64-pc-windows-msvc`
-
-Each native CI job builds and validates an MCPB 0.3 bundle. Release automation is tag-triggered.
-
 ## Release preparation
 
 - Keep the version synchronized in `Cargo.toml`, the root `tiingo-mcp` entry in `Cargo.lock`, and `packaging/mcpb/manifest.json`.
-- Add the matching `CHANGELOG.md` section as `X.Y.Z (Unreleased)` while changes are accumulating.
-- Before creating the release tag, replace `Unreleased` with the release date for changelog history.
-- Keep the GitHub Release title derived from the tag without a supported namespace or leading `v` so the webpage shows only `X.Y.Z`, with no date or timestamp.
-- Keep current-release URLs in `README.md` pointed at the latest published tag until the replacement release exists.
-- Run the full verification set plus `dist plan` and `cargo publish --dry-run --locked` before requesting release authorization.
+- Add the matching `CHANGELOG.md` section as `X.Y.Z (Unreleased)` while versioned release work accumulates. For this unversioned expansion, keep the top-level `Unreleased` section until a version is authorized.
+- Before creating a release tag, replace the versioned `Unreleased` marker with the release date.
+- Keep current-release URLs in `README.md` pointed at the latest published tag until its replacement exists.
+- Keep GitHub Release titles derived from the tag as bare `X.Y.Z`, with no supported namespace, date, or timestamp.
+- Run the [QUALITY.md](QUALITY.md) release gate plus `dist plan` and `cargo publish --dry-run --locked` before requesting release authorization.

API_SURFACE.md

@@ -0,0 +1,100 @@
+# API surface
+
+This is the maintained tool-to-upstream contract for the current 38-tool server. The original 17 tool names, required inputs, omission behavior, and results remain compatible with `tests/contract/baseline/v1-mcp.json`; four approved optional `columns` additions are the only legacy descriptor deltas, and every other row is additive.
+
+Status means:
+
+- **documented** — present in Tiingo's public endpoint documentation;
+- **beta** — documented by Tiingo as beta or early beta;
+- **vendor-supplied** — supplied for this integration but not in the public endpoint catalog; and
+- **lifecycle** — a finite local operation over an implemented upstream WebSocket service.
+
+Test/access classes are `D` (deterministic-testable), `L` (bounded read-only live-testable), `E` (entitlement or availability dependent), and `Q` (consumes live quota or bandwidth). These classes describe what can be tested safely; `L` does not claim that every row has its own checked-in ignored test. Every live call is also `Q`. Bulk and all-market operations have no routine live smoke.
+
+## Implemented tools
+
+| Tool | Tiingo route or upstream service | Status | Access and entitlement | Test/access class |
+|---|---|---|---|---|
+| `get_stock_metadata` | `GET /tiingo/daily/{ticker}` | documented | Key capability dependent | D, L, Q |
+| `get_stock_prices` | `GET /tiingo/daily/{ticker}/prices` | documented | Key capability dependent | D, L, Q |
+| `get_bulk_eod_prices` | `GET /tiingo/daily/prices?format=csv` | documented | Bulk response; typed JSON output; no routine live smoke | D, Q |
+| `get_ticker_metadata` | `GET /tiingo/daily/meta?columns=...` | vendor-supplied | Availability dependent; 1–32 allowlisted columns; no bulk live smoke | D, E, Q |
+| `get_realtime_price` | `GET /iex/{ticker}` | documented | Full TOPS fields require IEX entitlement | D, L, E, Q |
+| `get_intraday_prices` | `GET /iex/{ticker}/prices` | documented | Key capability dependent | D, L, Q |
+| `get_iex_market_snapshot` | `GET /iex` | documented | All-market response; IEX fields are entitlement dependent; no routine live smoke | D, E, Q |
+| `get_equity_realtime_snapshot` | `GET /tiingo/equity/intraday[/{ticker}]` | beta | Ticker-filtered live check only | D, L, Q |
+| `get_equity_intraday_prices` | `GET /tiingo/equity/intraday/{ticker}/prices` | beta | Consolidated 4am–8pm ET product | D, L, Q |
+| `get_boats_snapshot` | `GET /boats[/{ticker}]` | beta | Separate BOATS add-on; ticker live check only | D, L, E, Q |
+| `get_boats_prices` | `GET /boats/{ticker}/prices` | beta | Separate BOATS add-on, 8pm–3:59am ET | D, L, E, Q |
+| `get_fund_metadata` | `GET /tiingo/funds/{ticker}` | documented | Enterprise/institutional fund-fee capability | D, L, E, Q |
+| `get_fund_fee_metrics` | `GET /tiingo/funds/{ticker}/metrics` | documented | Enterprise/institutional fund-fee capability | D, L, E, Q |
+| `search_tiingo_assets` | `GET /tiingo/utilities/search?query=...` | beta | Early-beta response fields may change | D, L, Q |
+| `get_crypto_yield_platforms` | `GET /tiingo/crypto-yield/platforms` | documented | Plan/entitlement dependent | D, E, Q |
+| `get_crypto_yield_pools` | `GET /tiingo/crypto-yield/pools` | documented | Plan/entitlement dependent | D, E, Q |
+| `get_crypto_yield_ticks` | `GET /tiingo/crypto-yield/ticks` | documented | Plan/entitlement dependent | D, E, Q |
+| `get_crypto_yield_metrics` | `GET /tiingo/crypto-yield/{poolCode}/metrics` | documented | Plan/entitlement dependent | D, L, E, Q |
+| `get_forex_quote` | `GET /tiingo/fx/{ticker}/top` | beta | Key capability dependent | D, L, Q |
+| `get_forex_quotes` | `GET /tiingo/fx/top?tickers=...` | beta | 1–100 explicit pairs | D, L, Q |
+| `get_forex_prices` | `GET /tiingo/fx/{ticker}/prices` | beta | Key capability dependent | D, L, Q |
+| `get_crypto_quote` | `GET /tiingo/crypto/prices` | documented | Omit tickers only with bulk intent | D, L, Q |
+| `get_crypto_prices` | `GET /tiingo/crypto/prices` | documented | Use bounded ticker/date filters for live checks | D, L, Q |
+| `get_crypto_metadata` | `GET /tiingo/crypto` | documented | Filtered live check | D, L, Q |
+| `get_news` | `GET /tiingo/news` | documented | Dynamic content; use bounded filters | D, L, Q |
+| `get_fundamentals_definitions` | `GET /tiingo/fundamentals/definitions` | documented | Fundamentals entitlement dependent | D, L, E, Q |
+| `get_financial_statements` | `GET /tiingo/fundamentals/{ticker}/statements` | documented | Fundamentals entitlement dependent | D, L, E, Q |
+| `get_daily_fundamentals` | `GET /tiingo/fundamentals/{ticker}/daily` | documented | Fundamentals entitlement dependent | D, L, E, Q |
+| `get_company_meta` | `GET /tiingo/fundamentals/meta` | documented | Fundamentals entitlement dependent | D, E, Q |
+| `get_distributions_by_ex_date` | `GET /tiingo/corporate-actions/distributions?exDate=...` | beta | Early-release, entitlement-dependent; may include announced future actions | D, L, E, Q |
+| `get_dividends` | `GET /tiingo/corporate-actions/{ticker}/distributions` | beta | Early-release and entitlement-dependent | D, L, E, Q |
+| `get_dividend_yield` | `GET /tiingo/corporate-actions/{ticker}/distribution-yield` | beta | Early-release and entitlement-dependent | D, L, E, Q |
+| `get_splits` | `GET /tiingo/corporate-actions/{ticker}/splits` | beta | Early-release and entitlement-dependent | D, L, E, Q |
+| `get_splits_by_ex_date` | `GET /tiingo/corporate-actions/splits?exDate=...` | beta | Early-release, entitlement-dependent; may include announced/cancelled future actions | D, L, E, Q |
+| `start_market_data_subscription` | IEX `wss://api.tiingo.com/iex` or consolidated `wss://api.tiingo.com/equity/intraday` | lifecycle | IEX 6 default; 0/5 need direct-agreement confirmation. Consolidated accepts 4/6. | D, L, E, Q |
+| `poll_market_data_subscription` | Existing local subscription queue | lifecycle | Bounded cursor poll with sanitized terminal classification; no new upstream subscription | D, L, Q |
+| `update_market_data_subscription` | Upstream update using acknowledged subscription ID | lifecycle | Add/remove explicit symbols; partial failures report `appliedSymbols`; threshold changes require stop/start | D, L, Q |
+| `stop_market_data_subscription` | Best-effort upstream unsubscribe and local cleanup | lifecycle | Idempotent; never exposes upstream subscription ID | D, L, Q |
+
+## Checked-in ignored live smokes
+
+This inventory records the live harness that is actually checked in. It is intentionally narrower than the `L` capability classification above. Every row requires `TIINGO_API_KEY`, is ignored by default, and consumes quota or bandwidth.
+
+| Ignored test | Tools exercised | Bound |
+|---|---|---|
+| `live_boats_single_ticker` | `get_boats_snapshot`, `get_boats_prices` | One ticker; recent seven-day OHLCV history; entitlement-classifying |
+| `live_consolidated_equity_single_ticker` | `get_equity_realtime_snapshot`, `get_equity_intraday_prices` | One ticker; recent seven-day OHLCV history; documented session |
+| `live_consolidated_level_six_single_ticker_websocket` | `start_market_data_subscription`, `poll_market_data_subscription`, `stop_market_data_subscription` | Three level-6 one-ticker lifecycles; finite poll/cleanup |
+| `live_crypto_yield_metrics_single_pool` | `get_crypto_yield_metrics` | One pool/date range; three samples or early 403 |
+| `live_distributions_by_ex_date_tiny_filter` | `get_distributions_by_ex_date` | One exact ex-date; three samples or early 403 |
+| `live_forex_quotes_single_pair` | `get_forex_quotes` | One pair; three samples or early 403 |
+| `live_fund_fees_single_ticker` | `get_fund_metadata`, `get_fund_fee_metrics` | One ticker; three samples per operation or early 403 |
+| `live_iex_level_six_single_ticker_websocket` | `start_market_data_subscription`, `poll_market_data_subscription`, `stop_market_data_subscription` | Three level-6 one-ticker lifecycles; finite poll/cleanup |
+| `live_mcp_eod_data_is_consistent_accurate_and_timely` | `get_stock_prices` | One ticker/date; three MCP samples |
+| `live_read_only_tiingo_capabilities` | `get_stock_metadata`, `get_stock_prices`, `get_forex_quote`, `get_crypto_quote`, `get_news`, `get_fundamentals_definitions`, `get_dividends` | Representative bounded baseline calls |
+| `live_search_early_beta` | `search_tiingo_assets` | One query; three samples or early 403 |
+| `live_splits_by_ex_date_tiny_filter` | `get_splits_by_ex_date` | One exact ex-date; three samples or early 403 |
+
+## Audited but excluded or deferred
+
+| Surface | Classification | Reason |
+|---|---|---|
+| `GET /tiingo/crypto/top` | deprecated | Tiingo's current crypto prices route is implemented instead. |
+| News bulk downloads | deliberately excluded | Institutional, token-bearing download URLs and binary/unbounded payloads do not fit the bounded JSON tool contract. |
+| Company descriptions, security-master bulk, raw crypto/DEX/fundamental/yield bulk | deliberately excluded | Current products lack a stable public route contract suitable for this server. |
+| Small Exchange REST/WebSocket | deferred | Only a hidden legacy page exists; no published WebSocket URL/frame contract. |
+| Unified 24x5 equities | deliberately excluded | No single documented route exists; consolidated and BOATS remain separate products and sessions. |
+| BOATS WebSocket | deferred | Audited level-3 add-on, outside the approved upstream streaming slice. |
+| Crypto WebSocket | deferred | Audited thresholds 2/5, outside the approved upstream streaming slice. |
+| Forex WebSocket | deferred | Official threshold documentation conflicts; implementation waits for vendor clarification. |
+| Legacy test WebSocket | deliberately excluded | Old authentication syntax and no product contract. |
+
+## Official sources
+
+Audit date: **2026-08-25**. Entitlements and beta availability can change; current account behavior is authoritative.
+
+- [Tiingo API overview](https://www.tiingo.com/documentation/general/overview)
+- [End-of-Day](https://www.tiingo.com/documentation/end-of-day), the exact [bulk-ingest and corporate-action reseed workflow](https://www.tiingo.com/kb/article/the-fastest-method-to-ingest-tiingo-end-of-day-stock-api-data/), [IEX REST](https://www.tiingo.com/documentation/iex), [consolidated equity REST](https://www.tiingo.com/documentation/equity-realtime-stock-data), and [BOATS REST](https://www.tiingo.com/documentation/boats)
+- [Forex](https://www.tiingo.com/documentation/forex), [crypto](https://www.tiingo.com/documentation/crypto), [Crypto Yield](https://www.tiingo.com/documentation/crypto-yield), [news](https://www.tiingo.com/documentation/news), and [Search](https://www.tiingo.com/documentation/utilities/search)
+- [Fundamentals](https://www.tiingo.com/documentation/fundamentals), [fund fees](https://www.tiingo.com/documentation/mutual-fund-and-etf-fees), [dividends](https://www.tiingo.com/documentation/corporate-actions/dividends), and [splits](https://www.tiingo.com/documentation/corporate-actions/splits)
+- [IEX WebSocket](https://www.tiingo.com/documentation/websockets/iex) and [consolidated equity WebSocket](https://www.tiingo.com/documentation/websockets/equity-realtime-stock-data)
+
+See [ARCHITECTURE.md](ARCHITECTURE.md) for lifecycle bounds and [QUALITY.md](QUALITY.md) for evidence requirements.