Skip to content

kit sql (JDBC) unreachable on SOCKS-only clusters — DriverManager connects to private IP without SOCKS proxy #735

Description

@pmcfadin

Summary

easy-db-lab <kit> sql ... (JDBC capability) cannot reach the target on a SOCKS-proxy-only cluster (no Tailscale). Every query fails with java.net.SocketException: Connection reset, including a trivial SELECT 1. This blocks all JDBC-based verification (Trino sql, and any other kit exposing an EndpointType.JDBC capability) whenever the cluster was brought up without Tailscale.

Root cause

KitJdbcSqlService.execute() resolves the endpoint to the node's private IP and connects via DriverManager.getConnection("jdbc:trino://10.1.x.x:8080", ...) (services/sql/KitJdbcSqlService.kt + SqlQueryService.kt:52-56, URL from KitConfig.formatUrljdbc:$scheme://$ip:$port$path).

On a SOCKS-only cluster the private IP is only reachable through the ssh -N -D SOCKS tunnel. But ProcessSocksProxyService deliberately does not set the global socksProxyHost/socksProxyPort system properties (documented in its class comment — doing so would route the AWS SDK/S3 through the tunnel and break direct AWS access on corporate networks). Instead, "the clients that need the tunnel (fabric8 K8s, OkHttp) read the port and configure the SOCKS proxy explicitly."

The JDBC path is a client that needs the tunnel but does not configure SOCKS explicitly — so DriverManager opens a direct socket to a private IP that isn't routable from the developer's machine → connection reset. On a Tailscale cluster the private IP is directly routable, which is why this has been invisible.

Reproduce

  1. init ... --up without Tailscale (SOCKS-only; state.json shows tailscaleActive: false).
  2. Install + start the trino kit.
  3. easy-db-lab trino sql "SELECT 1"Error executing query: java.net.SocketException: Connection reset.

Confirmation / workaround

Forcing the JVM's global SOCKS proxy makes it work:

JAVA_TOOL_OPTIONS="-DsocksProxyHost=localhost -DsocksProxyPort=1080 -DsocksProxyVersion=5" \
  easy-db-lab trino sql "SELECT 1"     # → returns 1 row

(Global props are exactly what the proxy service avoids process-wide, so this is a diagnostic workaround, not the fix.)

Suggested fix

Configure SOCKS per-connection on the JDBC path only — mirroring how fabric8/OkHttp already do it — so AWS SDK sockets are unaffected:

  • Trino JDBC supports socksProxy=host:port as a connection property / URL param. When a SOCKS proxy is active (and Tailscale is not), append socksProxy=localhost:<proxyPort> to the Trino JDBC URL (or set it in the Properties passed to getConnection).
  • More generally, KitJdbcSqlService / defaultJdbcConnectionFactory should consult the active SocksProxyState (via Constants.Proxy.PORT_PROPERTY) and set the driver-appropriate SOCKS connection property when !tailscaleActive. Each JDBC driver has its own knob (Trino: socksProxy; pgjdbc: socksProxyHost/socksProxyPort; MySQL: via socksProxyHost), so this likely belongs behind the endpoint/driver abstraction.

Reproduced on a real 3-node AWS cluster during cqlite epic #2103 round-4 lab testing (clusters/cqlite-flight-r4-20260708-160449). Not a blocker for the lab run (workaround applied) but it makes <kit> sql unusable on any SOCKS-only cluster.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions