feat(csharp): add enable_fast_metadata_query flag for DESC TABLE EXTENDED

[jadewang-db]

Summary

Adds opt-in adbc.databricks.enable_fast_metadata_query flag (default false) that pairs the new SQL modifier STATIC ONLY (runtime PR #198486) on DESC TABLE EXTENDED AS JSON with the matching off-WLM routing signal per protocol.

Both signals are required together — STATIC ONLY alone still queues through WLM; off-WLM routing alone still does the full metadata scan.

Protocol	SQL emitted	Off-WLM signal	Conditions
Thrift	DESC TABLE EXTENDED <t> STATIC ONLY AS JSON	RunAsync=false on descStmt	flag set AND warehouse path (/sql/1.0/warehouses/{id} or /sql/1.0/endpoints/{id}). General-cluster paths: flag is ignored.
SEA	DESC TABLE EXTENDED <t> STATIC ONLY AS JSON	x-databricks-sea-can-run-fully-sync: true (already sent by ExecuteMetadataSqlAsync)	flag set. SEA is always-warehouse.

Implementation

• DatabricksConnection.IsWarehousePath matches the path regex (cached) — only used to gate the Thrift opt-in.
• DatabricksStatement.RunAsyncOverride is a new internal nullable bool; when non-null it takes precedence over the connection-level RunAsyncInThrift inside SetStatementProperties. The override is set on the per-call descStmt in GetColumnsExtendedAsync rather than mutating connection state.
• StatementExecutionConnection.EnableFastMetadataQuery parses the same flag for SEA.

Fallback safety

STATIC ONLY is rejected with parse error 42601 (INVALID_STATIC_ONLY_USAGE) on servers without PR #198486. The existing catch (HiveServer2Exception ex) when (ex.SqlState == "42601" || ex.SqlState == "20000") in GetColumnsExtendedAsync already handles that — falls back to GetColumns + GetPrimaryKeys + GetCrossReference.

Test plan

• Added unit tests for the new parameter constant.
• Added unit tests for IsWarehousePath covering /sql/1.0/warehouses/{id}, /sql/1.0/endpoints/{id} (with/without trailing slash, with/without query strings), /sql/protocolv1/... clusters, empty path, and the AdbcOptions.Uri fallback.
• Added unit tests for UseFastMetadataQuery covering all four combinations of (flag, warehouse-path).
• Full dotnet test --filter "FullyQualifiedName~Unit." suite: 638 passed, 0 failed.
• E2E verification against a DBSQL warehouse running PR #198486 is gated on that runtime change reaching the test env; existing 42601 fallback protects pre-rollout.

Closes PECO-3021

This pull request and its description were written by Isaac.

[jadewang-db]

Self-review

Overall the change is small and well-scoped. The Thrift-side opt-in plus warehouse-path gating matches the JIRA, and SEA was correctly identified as already wired via x-databricks-sea-can-run-fully-sync. One high-priority PR-hygiene issue around the branch state, plus a handful of low-priority simplifications inline.

Positive observations

• UseFastMetadataQuery = _enableFastMetadataQuery && IsWarehousePath is the right composition; unit tests enumerate the full truth table.
• Per-statement RunAsyncOverride is the right surface — mutating connection state would have been worse.
• Existing 42601/20000 fallback in GetColumnsExtendedAsync still protects against servers that reject the new shape, so rollout is safe.
• Activity tag fast_metadata_query will make the flag's effect easy to confirm in production traces.

HIGH — branch is significantly behind main

The PR diff lists 11 files (+277/-9), but the commit f8f66af only touches 6 files (+200/-3). The extras come from telemetry refactors that landed on main while this branch was idle (e.g., PECO-2978/2988/2992 in DatabricksStatement.cs and ConnectionTelemetry.cs), plus d02dae2 which was already merged via #401.

$ git log origin/main..PECO-3021 --oneline
f8f66af feat(csharp): add enable_fast_metadata_query flag for DESC TABLE EXTENDED
d02dae2 fix(csharp): emit bare hostname for telemetry host_url to match JDBC   ← already merged via #401

Recommend git rebase origin/main and force-push so the diff cleanly reflects only PECO-3021's work.

Test coverage note

Coverage of new code is exclusively unit-level. Wire behavior (server actually receiving RunAsync=false) is not asserted — relies on the existing E2E harness once runtime PR #198486 deploys. Worth mentioning the intentional E2E gap in the PR body.

[jadewang-db]

Self-review pass on PECO-3021. Overall the design is clean — the AND-gate of flag + warehouse-path on Thrift, the per-statement RunAsyncOverride to avoid mutating connection state, and the reuse of the existing 42601/20000 fallback for old servers are all the right shapes. The unit-test matrix on IsWarehousePath and UseFastMetadataQuery is thorough.

Findings below are mostly Low/Medium polish — no Critical or High issues. One scope/PR-hygiene note: this PR also carries commit d02dae2 (telemetry host_url bare-hostname fix, PECO-2987) which the description doesn't mention. Consider either splitting it out or extending the PR body so reviewers know both changes are in flight.

[eric-wang-1990]

Why do we need this extra flag? Who is passing this in?
Can we just rely on runAsyncInThrift?

[jadewang-db]

runAsyncInThrift is the user's connection-level setting (adbc.databricks.enable_run_async_in_thrift_op, defaults to true) — flipping it on the connection would affect all their statements, and mutating it around the desc-stmt call would race with any concurrent statements on the same connection.

RunAsyncOverride is an internal per-statement mechanism — no user surface — used exclusively by GetColumnsExtendedAsync to flip RunAsync=false on its own internal descStmt while the user's other statements keep their configured setting. The pairing with the STATIC ONLY SQL modifier is what tells the warehouse to route the desc-stmt off the WLM path; the connection-level flag alone can't express "this single statement, no others."

Open to a different shape if you have one in mind — happy to refactor if e.g. you'd prefer a constructor param or a dedicated ExecuteWithRunAsyncOff() helper instead of the override property.

[msrathore-db]

LGTM apart from Eric's comment