fix: avoid re-running run-once migrations (409)#422
fix: avoid re-running run-once migrations (409)#422cburyta wants to merge 3 commits intoSnowflake-Labs:masterfrom
Conversation
…atch (Snowflake-Labs#409) Snowflake stores schema and table names in uppercase in INFORMATION_SCHEMA. When change_history_table is specified with lowercase names (e.g. from config), the previous REPLACE-only comparison failed, so fetch_change_history_metadata() returned {} and schemachange re-ran all versioned scripts. - fetch_change_history_metadata: UPPER(REPLACE(...)) for TABLE_SCHEMA and TABLE_NAME - change_history_schema_exists: UPPER(REPLACE(...)) for SCHEMA_NAME
|
Thanks for the PR! One thing to flag on the Rather than changing the SQL, would you be open to fixing this in elif snowflake_identifier_pattern.match(input_value):
return input_value.upper()That handles both cases correctly without needing any changes to the queries. |
|
Glad to go that route editing in the python side - that would feel cleaner. However a couple notes...
DeployConfig.py might change the case of other props for example. (Edit: I might consider editing ChangeHistoryTable.py from_str for a python based approach.) |
Pull Request: Fix INFORMATION_SCHEMA case sensitivity (#409)
Summary
Fixes the bug where schemachange re-deploys all versioned scripts when the change history table is specified with lowercase schema/table names (e.g.
mydb.myschema.change_history). Snowflake stores identifiers in uppercase inINFORMATION_SCHEMA; the previous query used a case-sensitive comparison, so the lookup returned no rows and schemachange treated the table as missing.Problem
fetch_change_history_metadata()andchange_history_schema_exists()compared againstINFORMATION_SCHEMAusingREPLACE('schema_name','"','')without normalizing case. Snowflake returnsTABLE_SCHEMA = 'MYSCHEMA', so'myschema' != 'MYSCHEMA'and the query returns no rows.Solution
Use
UPPER(REPLACE(...))in the WHERE clause for:fetch_change_history_metadata()–TABLE_SCHEMAandTABLE_NAMEin theINFORMATION_SCHEMA.TABLESquery.change_history_schema_exists()–SCHEMA_NAMEin theINFORMATION_SCHEMA.SCHEMATAquery.No change to object references (e.g.
fully_qualified); Snowflake is already case-insensitive for those. Only the INFORMATION_SCHEMA lookups are fixed.Testing
test_fetch_change_history_metadata_uses_upper_for_information_schema– builds a session with lowercaseChangeHistoryTableand asserts the executed query containsUPPER(REPLACE(so the lookup is case-insensitive.Checklist
pytest)ruff format .)ruff check .)