Skip to content

GetObjects at Columns depth runs SHOW COLUMNS IN ACCOUNT (80-150s) for underscore catalog names #169

Description

@zeroshade

Summary

GetObjects at ObjectDepthColumns / ObjectDepthAll can take 80–150 seconds for a single-table request when the catalog (database) name contains an underscore.

Reported on #105, but the root cause is unrelated to that PR.

Symptoms

Calling GetObjects(depth=Columns, catalog="DB_NAME", dbSchema="TEST_SCHEMA", tableName="LINEITEM") takes ~90–150s, regardless of how many columns the target table has (reproduced with a 1-column table). The equivalent call was ~10s with the older apache/arrow-adbc/go/adbc driver.

QUERY_HISTORY from a ~92s call shows the time is almost entirely one query:

SHOW query (/* ADBC:getObjects */) Elapsed
SHOW TERSE DATABASES LIKE 'DB_NAME' IN ACCOUNT 179 ms
SHOW UNIQUE KEYS ... IN ACCOUNT 406 ms
SHOW IMPORTED KEYS ... IN ACCOUNT 267 ms
SHOW COLUMNS /* ADBC:getObjects */ IN ACCOUNT 84,866 ms
SHOW TERSE SCHEMAS LIKE 'TEST_SCHEMA' IN ACCOUNT 333 ms

Root cause

GetObjects builds the scope suffix for SHOW COLUMNS from isWildcardStr, which treats both _ and % as wildcards:

func isWildcardStr(ident string) bool {
    return strings.ContainsAny(ident, "_%")
}

For an ordinary database name like DB_NAME, the catalog is considered a wildcard, so the scope collapses to IN ACCOUNT. Unlike the other SHOW commands, SHOW COLUMNS has no table-level LIKE filter, so SHOW COLUMNS ... IN ACCOUNT scans every column of every table in every database in the account. On a large account that is the 85s above; the target table's column count is irrelevant.

The SHOW COLUMNS call was introduced in #152 (fix(go): set xdbc_column_size for BINARY columns in GetObjects) to read byteLength for BINARY column sizes. The older apache/arrow-adbc driver read column metadata solely from information_schema.columns (scoped by catalog in SQL) and never issued a SHOW COLUMNS, which is why the regression appears when moving to this driver.

Scope timing measured on a small test account (worse on large accounts):

SHOW COLUMNS scope Elapsed
IN ACCOUNT 18.4 s
IN DATABASE 4.8 s
IN SCHEMA 0.52 s
IN TABLE 0.39 s

Proposed fix

Scope the SHOW COLUMNS query to the narrowest concrete object (IN TABLE / IN SCHEMA / IN DATABASE), treating _ as a literal and only % / .* / nil as broadening. This is safe because SHOW COLUMNS is only a best-effort source for BINARY xdbc_column_size; the authoritative column list still comes from information_schema.columns filtered by ILIKE, so a narrower scope never drops columns or changes non-BINARY metadata.

Tradeoff: a BINARY column reached only through a genuine _-as-wildcard match may report a NULL xdbc_column_size, matching the behavior before #152.

Environment

  • Driver: github.com/adbc-drivers/snowflake/go with github.com/snowflakedb/gosnowflake/v2
  • Previously fast: github.com/apache/arrow-adbc/go/adbc (v1.11.0) with github.com/snowflakedb/gosnowflake (v1.19.1)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglang-goThe Go Snowflake driver.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions