Skip to content

Commit 8de6ebf

Browse files
alan-andradeclaude
andauthored
fix: Tune timeouts on dbt platform httpx clients (#803)
## Summary - Set explicit timeouts on all `httpx.AsyncClient()` calls that hit dbt platform APIs - Timeout values based on production latency data (Datadog spans, last 7 days) - Tiered by API latency profile rather than one-size-fits-all ## Why `httpx.AsyncClient()` defaults to a 5-second timeout. For Admin API calls like `GET /runs/{id}/`, the p99 latency is 5.0s, meaning ~1% of requests hit the timeout. This caused `ReadTimeout` errors in production. **82 errors over the last 14 days, accelerating**: 5-6/day in week 1, 10-17/day in week 2. The last 3 full days (Jun 8-10) accounted for 50% of all errors. Top 3 tools by error count: `list_metrics` (22), `get_dimensions` (20), `get_job_run_error` (19), accounting for 74% of all timeouts. A single account (46429) produced 46% of all errors. ## Timeout values | Client | Old timeout | New timeout | Justification | |---|---|---|---| | Admin API (`dbt_admin/client.py`) | 5s (default) | **15s** | p99 = 5.0s, max = 5.3s; 15s gives ~3x headroom | | Discovery API (`discovery/client.py`) | 5s (default) | **15s** | p99 = 3.8s (worst cluster), max = 77s (outlier); 15s covers normal operation | | Project/env resolvers | 5s (default) | **15s** | Same platform API endpoints as Admin API | | Semantic Layer GQL (`gql_request.py`) | 30s | **60s** | Schema operations reach 38.7s max; 60s gives ~1.5x headroom | Product docs clients (fetching from external docs sites) retain their existing explicit timeouts (30s, 120s). ## Test plan - [x] `ruff check` and `ruff format` pass - [x] 748 unit tests pass (rebased on latest main) - [ ] Verify ReadTimeout error rate drops in production after deploy 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent d32acf3 commit 8de6ebf

6 files changed

Lines changed: 16 additions & 7 deletions

File tree

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
kind: Bug Fix
2+
body: Set explicit timeouts on all dbt platform httpx clients to prevent ReadTimeout errors (15s for REST APIs, 60s for Semantic Layer GQL)
3+
time: 2026-06-11T14:24:37.725211+01:00

src/dbt_mcp/config/settings.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@
1515
logger = logging.getLogger(__name__)
1616

1717
DEFAULT_DBT_CLI_TIMEOUT = 60
18+
PLATFORM_API_TIMEOUT = 15.0
19+
SEMANTIC_LAYER_GQL_TIMEOUT = 60.0
1820

1921

2022
class DbtMcpLogSettings(BaseSettings):

src/dbt_mcp/dbt_admin/client.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
import httpx
88

99
from dbt_mcp.config.config_providers import AdminApiConfig, ConfigProvider
10+
from dbt_mcp.config.settings import PLATFORM_API_TIMEOUT
1011
from dbt_mcp.errors import (
1112
AdminAPIError,
1213
ArtifactRetrievalError,
@@ -43,7 +44,7 @@ async def _make_request(
4344
headers = await self.get_headers()
4445

4546
try:
46-
async with httpx.AsyncClient() as client:
47+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
4748
response = await client.request(
4849
method,
4950
url,
@@ -394,7 +395,7 @@ async def get_job_run_artifact(
394395
} | config.headers_provider.get_headers()
395396

396397
try:
397-
async with httpx.AsyncClient() as client:
398+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
398399
response = await client.get(
399400
f"{config.url}/api/v2/accounts/{account_id}/runs/{run_id}/artifacts/{artifact_path}",
400401
headers=get_artifact_header,

src/dbt_mcp/discovery/client.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
from pydantic import BaseModel, ConfigDict, Field
88

99
from dbt_mcp.config.config_providers import DiscoveryConfig
10+
from dbt_mcp.config.settings import PLATFORM_API_TIMEOUT
1011
from dbt_mcp.discovery.graphql import load_query
1112
from dbt_mcp.errors import InvalidParameterError, ToolCallError
1213
from dbt_mcp.errors.common import NotFoundError
@@ -362,7 +363,7 @@ async def execute_query(
362363
url = config.url
363364
headers = config.headers_provider.get_headers()
364365

365-
async with httpx.AsyncClient() as client:
366+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
366367
response = await client.post(
367368
url=url,
368369
json={"query": query, "variables": variables},

src/dbt_mcp/project/project_resolver.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
import httpx
66

7+
from dbt_mcp.config.settings import PLATFORM_API_TIMEOUT
78
from dbt_mcp.oauth.dbt_platform import (
89
DbtPlatformAccount,
910
DbtPlatformProject,
@@ -18,7 +19,7 @@ async def get_account(
1819
account_id: int,
1920
headers: dict[str, str],
2021
) -> DbtPlatformAccount:
21-
async with httpx.AsyncClient() as client:
22+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
2223
response = await client.get(
2324
# Using v2 as this endpoint in v3 was not available in testing
2425
url=f"{dbt_platform_url}/api/v2/accounts/{account_id}/",
@@ -33,7 +34,7 @@ async def get_all_accounts(
3334
dbt_platform_url: str,
3435
headers: dict[str, str],
3536
) -> list[DbtPlatformAccount]:
36-
async with httpx.AsyncClient() as client:
37+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
3738
response = await client.get(
3839
url=f"{dbt_platform_url}/api/v3/accounts/",
3940
headers=headers,
@@ -53,7 +54,7 @@ async def get_all_projects_for_account(
5354
"""Fetch all projects for an account using offset/page_size pagination."""
5455
offset = 0
5556
projects: list[DbtPlatformProject] = []
56-
async with httpx.AsyncClient() as client:
57+
async with httpx.AsyncClient(timeout=PLATFORM_API_TIMEOUT) as client:
5758
while True:
5859
response = await client.get(
5960
f"{dbt_platform_url}/api/v3/accounts/{account.id}/projects/?state=1&offset={offset}&limit={page_size}",

src/dbt_mcp/semantic_layer/gql/gql_request.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
11
import httpx
22

33
from dbt_mcp.config.config_providers import SemanticLayerConfig
4+
from dbt_mcp.config.settings import SEMANTIC_LAYER_GQL_TIMEOUT
45
from dbt_mcp.gql.errors import raise_gql_error
56

67

78
async def submit_request(
89
sl_config: SemanticLayerConfig,
910
payload: dict,
10-
timeout: float = 30.0,
11+
timeout: float = SEMANTIC_LAYER_GQL_TIMEOUT,
1112
) -> dict:
1213
if "variables" not in payload:
1314
payload["variables"] = {}

0 commit comments

Comments
 (0)