Summary
GetObjects at ObjectDepthColumns / ObjectDepthAll can take 80–150 seconds for a single-table request when the catalog (database) name contains an underscore.
Reported on #105, but the root cause is unrelated to that PR.
Symptoms
Calling GetObjects(depth=Columns, catalog="DB_NAME", dbSchema="TEST_SCHEMA", tableName="LINEITEM") takes ~90–150s, regardless of how many columns the target table has (reproduced with a 1-column table). The equivalent call was ~10s with the older apache/arrow-adbc/go/adbc driver.
QUERY_HISTORY from a ~92s call shows the time is almost entirely one query:
SHOW query (/* ADBC:getObjects */) |
Elapsed |
SHOW TERSE DATABASES LIKE 'DB_NAME' IN ACCOUNT |
179 ms |
SHOW UNIQUE KEYS ... IN ACCOUNT |
406 ms |
SHOW IMPORTED KEYS ... IN ACCOUNT |
267 ms |
SHOW COLUMNS /* ADBC:getObjects */ IN ACCOUNT |
84,866 ms |
SHOW TERSE SCHEMAS LIKE 'TEST_SCHEMA' IN ACCOUNT |
333 ms |
Root cause
GetObjects builds the scope suffix for SHOW COLUMNS from isWildcardStr, which treats both _ and % as wildcards:
func isWildcardStr(ident string) bool {
return strings.ContainsAny(ident, "_%")
}
For an ordinary database name like DB_NAME, the catalog is considered a wildcard, so the scope collapses to IN ACCOUNT. Unlike the other SHOW commands, SHOW COLUMNS has no table-level LIKE filter, so SHOW COLUMNS ... IN ACCOUNT scans every column of every table in every database in the account. On a large account that is the 85s above; the target table's column count is irrelevant.
The SHOW COLUMNS call was introduced in #152 (fix(go): set xdbc_column_size for BINARY columns in GetObjects) to read byteLength for BINARY column sizes. The older apache/arrow-adbc driver read column metadata solely from information_schema.columns (scoped by catalog in SQL) and never issued a SHOW COLUMNS, which is why the regression appears when moving to this driver.
Scope timing measured on a small test account (worse on large accounts):
SHOW COLUMNS scope |
Elapsed |
IN ACCOUNT |
18.4 s |
IN DATABASE |
4.8 s |
IN SCHEMA |
0.52 s |
IN TABLE |
0.39 s |
Proposed fix
Scope the SHOW COLUMNS query to the narrowest concrete object (IN TABLE / IN SCHEMA / IN DATABASE), treating _ as a literal and only % / .* / nil as broadening. This is safe because SHOW COLUMNS is only a best-effort source for BINARY xdbc_column_size; the authoritative column list still comes from information_schema.columns filtered by ILIKE, so a narrower scope never drops columns or changes non-BINARY metadata.
Tradeoff: a BINARY column reached only through a genuine _-as-wildcard match may report a NULL xdbc_column_size, matching the behavior before #152.
Environment
- Driver:
github.com/adbc-drivers/snowflake/go with github.com/snowflakedb/gosnowflake/v2
- Previously fast:
github.com/apache/arrow-adbc/go/adbc (v1.11.0) with github.com/snowflakedb/gosnowflake (v1.19.1)
Summary
GetObjectsatObjectDepthColumns/ObjectDepthAllcan take 80–150 seconds for a single-table request when the catalog (database) name contains an underscore.Reported on #105, but the root cause is unrelated to that PR.
Symptoms
Calling
GetObjects(depth=Columns, catalog="DB_NAME", dbSchema="TEST_SCHEMA", tableName="LINEITEM")takes ~90–150s, regardless of how many columns the target table has (reproduced with a 1-column table). The equivalent call was ~10s with the olderapache/arrow-adbc/go/adbcdriver.QUERY_HISTORYfrom a ~92s call shows the time is almost entirely one query:/* ADBC:getObjects */)SHOW TERSE DATABASES LIKE 'DB_NAME' IN ACCOUNTSHOW UNIQUE KEYS ... IN ACCOUNTSHOW IMPORTED KEYS ... IN ACCOUNTSHOW COLUMNS /* ADBC:getObjects */ IN ACCOUNTSHOW TERSE SCHEMAS LIKE 'TEST_SCHEMA' IN ACCOUNTRoot cause
GetObjectsbuilds the scope suffix forSHOW COLUMNSfromisWildcardStr, which treats both_and%as wildcards:For an ordinary database name like
DB_NAME, the catalog is considered a wildcard, so the scope collapses toIN ACCOUNT. Unlike the other SHOW commands,SHOW COLUMNShas no table-levelLIKEfilter, soSHOW COLUMNS ... IN ACCOUNTscans every column of every table in every database in the account. On a large account that is the 85s above; the target table's column count is irrelevant.The
SHOW COLUMNScall was introduced in #152 (fix(go): set xdbc_column_size for BINARY columns in GetObjects) to readbyteLengthfor BINARY column sizes. The olderapache/arrow-adbcdriver read column metadata solely frominformation_schema.columns(scoped by catalog in SQL) and never issued aSHOW COLUMNS, which is why the regression appears when moving to this driver.Scope timing measured on a small test account (worse on large accounts):
SHOW COLUMNSscopeIN ACCOUNTIN DATABASEIN SCHEMAIN TABLEProposed fix
Scope the
SHOW COLUMNSquery to the narrowest concrete object (IN TABLE/IN SCHEMA/IN DATABASE), treating_as a literal and only%/.*/ nil as broadening. This is safe becauseSHOW COLUMNSis only a best-effort source for BINARYxdbc_column_size; the authoritative column list still comes frominformation_schema.columnsfiltered byILIKE, so a narrower scope never drops columns or changes non-BINARY metadata.Tradeoff: a BINARY column reached only through a genuine
_-as-wildcard match may report a NULLxdbc_column_size, matching the behavior before #152.Environment
github.com/adbc-drivers/snowflake/gowithgithub.qkg1.top/snowflakedb/gosnowflake/v2github.qkg1.top/apache/arrow-adbc/go/adbc(v1.11.0) withgithub.qkg1.top/snowflakedb/gosnowflake(v1.19.1)