SNOW-3713887: Quote stage refs in uploadStream/downloadStream for non-ASCII names (#2683)

sfc-gh-dheyman · web-flow · commit b448bd537d14 · 2026-06-29T14:30:54.000+02:00
### TL;DR Fixed `uploadStream`/`downloadStream` failing with SQL compilation errors when stage references contain non-ASCII characters (e.g. Japanese schema names). ### What changed? Stage references in internally generated PUT/GET commands are now wrapped in single quotes when they contain characters that require quoting per Snowflake SQL syntax. Two new static helpers, `needsStageQuoting` and `quoteStageRefIfNeeded`, were added to `SnowflakeConnectionImpl` to determine whether a stage reference needs quoting and to apply it. Safe characters (ASCII alphanumerics and `_$./@~%"-:`) are left unquoted; anything else (spaces, non-ASCII/unicode, etc.) triggers single-quote wrapping with embedded single quotes doubled. Already-quoted references (single-quote or dollar-quote delimited) are returned unchanged. The `downloadStream` path was also refactored to build the stage reference separately before quoting, replacing the previous ad-hoc special-character check that only considered the file name portion and missed non-ASCII characters in the stage/schema name itself. ### How to test? - `StageQuotingTest` — new unit tests covering `needsStageQuoting` and `quoteStageRefIfNeeded` for ASCII refs, non-ASCII refs, spaces, already-quoted refs, embedded single quotes, and null inputs. - `StreamLatestIT.testUploadDownloadStreamWithUnicodeSchemaStage` — new integration test that creates a schema with a Japanese Unicode name, uploads a file via `uploadStream`, downloads it via `downloadStream`, and asserts the content round-trips correctly. ### Why make this change? `uploadStream` and `downloadStream` construct PUT/GET SQL commands internally. When a stage reference included non-ASCII characters (e.g. a Japanese schema name), the generated SQL was syntactically invalid, causing Snowflake to return a SQL compilation error. Wrapping such references in single quotes makes the generated SQL valid regardless of the characters present in the schema or stage name (SNOW-3713887).
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,6 +9,7 @@
   - Fixed flaky `DatabaseMetaDataLatestIT.testUnderscoreInSchemaNamePatternForPrimaryAndForeignKeys[WithPatternSearchDisabled]`: the SHOW-based `getPrimaryKeys`/`getImportedKeys` lookups could transiently return fewer rows than expected right after the table DDL (eventually-consistent PK/FK constraint metadata), so the lookups are now retried until the expected rows become visible. Test-only change with no driver behavior impact (snowflakedb/snowflake-jdbc#2673).
   - Fixed flaky `DatabaseMetaDataIT.testGetPrimarykeys` and `SnowflakeDriverIT.testConstraints` (eventually-consistent PK/FK constraint metadata after DDL) by retrying the constraint metadata lookups until visible. Replaced the live, concurrent stress tests `DatabaseMetaDataLatestIT.test[No]PatternSearchAllowedForPrimaryAndForeignKeys` (which timed out on slow CI runners) with deterministic coverage in the new `DatabaseMetadataWiremockLatestIT`, which validates the `getTables`/`getPrimaryKeys`/`getImportedKeys` SHOW-result mapping and the `enablePatternSearch` pattern-vs-literal behavior using stubbed responses (no live account or metadata-propagation dependency). Test-only change with no driver behavior impact (snowflakedb/snowflake-jdbc#2673).
   - Fixed the Loader API (`StreamLoader.setVectorColumns`) throwing `Loader$ConnectionError: ... Result set has been closed` when the `getColumns` metadata lookup returned no rows. The result set returned by `getColumns` closes itself once `next()` runs out of matching rows, so the unchecked `rs.next()` followed by `rs.getString(...)` raised "Result set has been closed"; the result is now read only when `rs.next()` returns a row. This intermittently aborted concurrent loads (e.g. `FlatfileReadMultithreadIT`) when the `SHOW COLUMNS` metadata query raced with concurrent DML on the same table (snowflakedb/snowflake-jdbc#2674).
+  - Fixed `uploadStream`/`downloadStream` failing with SQL compilation errors when the stage reference contains non-ASCII characters (e.g. Japanese schema names). Stage references in internally generated PUT/GET commands are now wrapped in single quotes when they contain characters that require quoting per Snowflake SQL syntax (SNOW-3713887).
   
 - v4.3.1
   - Fixed GCS-backed internal stage PUT failing with opaque `invalid_gcs_credentials` in SPCS pods on GCP: the GCS SDK's Application Default Credentials (ADC) probe was reaching out to `metadata.google.internal` which is unreachable inside SPCS; explicit credentials are now always set when a `GCS_ACCESS_TOKEN` is present, suppressing the ADC probe entirely. Also fixed `GCSAccessStrategyAwsSdk` rejecting custom GCS endpoints that lack an `https://` scheme prefix (e.g. bare `storage.me-central2.rep.googleapis.com`), mirroring the existing handling in `GCSDefaultAccessStrategy`. The catch-all in `setupGCSClient` now chains and logs the original exception instead of swallowing it (snowflakedb/snowflake-jdbc#2664).
diff --git a/src/main/java/net/snowflake/client/internal/api/implementation/connection/SnowflakeConnectionImpl.java b/src/main/java/net/snowflake/client/internal/api/implementation/connection/SnowflakeConnectionImpl.java
@@ -976,13 +976,7 @@ private void uploadStreamInternal(
 
     SnowflakeStatementImpl stmt = this.createStatement().unwrap(SnowflakeStatementImpl.class);
 
-    StringBuilder destStage = new StringBuilder();
-
-    // add stage name
-    if (!(stageName.startsWith("@") || stageName.startsWith("'@") || stageName.startsWith("$$@"))) {
-      destStage.append("@");
-    }
-    destStage.append(stageName);
+    StringBuilder destStage = new StringBuilder(normalizeStageNameForRef(stageName));
 
     // add dest prefix
     if (destPrefix != null) {
@@ -992,15 +986,18 @@ private void uploadStreamInternal(
       destStage.append(destPrefix);
     }
 
+    String destStageStr = destStage.toString();
+    String quotedDestStage = quoteStageRefIfNeeded(destStageStr);
+
     StringBuilder putCommand = new StringBuilder();
     // use a placeholder for source file
     putCommand.append("put file:///tmp/placeholder ");
-    putCommand.append(destStage.toString());
+    putCommand.append(quotedDestStage);
     putCommand.append(" overwrite=true");
 
     SFBaseFileTransferAgent transferAgent =
         sfConnectionHandler.getFileTransferAgent(putCommand.toString(), stmt.getSFBaseStatement());
-    transferAgent.setDestStagePath(destStage.toString());
+    transferAgent.setDestStagePath(destStageStr);
     transferAgent.setSourceStream(inputStream);
     transferAgent.setDestFileNameForStreamSource(destFileName);
     transferAgent.setCompressSourceFromStream(compressData);
@@ -1049,30 +1046,12 @@ public InputStream downloadStream(
             ResultSet.CONCUR_READ_ONLY,
             ResultSet.CLOSE_CURSORS_AT_COMMIT);
 
-    StringBuilder getCommand = new StringBuilder();
+    String stageRef = buildStageFileRef(stageName, sourceFileName);
+    String quotedStageRef = quoteStageRefIfNeeded(stageRef);
 
+    StringBuilder getCommand = new StringBuilder();
     getCommand.append("get ");
-
-    if (!stageName.startsWith("@")) {
-      getCommand.append("@");
-    }
-
-    getCommand.append(stageName);
-
-    getCommand.append("/");
-
-    if (sourceFileName.startsWith("/")) {
-      sourceFileName = sourceFileName.substring(1);
-    }
-
-    getCommand.append(sourceFileName);
-
-    // special characters and spaces require single quotes around stage name.
-    boolean isSpecialChar = !sourceFileName.matches("^[a-zA-Z0-9_/.]*$");
-    if (isSpecialChar) {
-      getCommand.insert(getCommand.indexOf("@"), "'");
-      getCommand.append("'");
-    }
+    getCommand.append(quotedStageRef);
 
     // this is a fake path, used to form Get query and retrieve stage info,
     // no file will be downloaded to this location
@@ -1174,4 +1153,107 @@ public void removeClosedStatement(Statement stmt) {
   private static String escapeIdentifier(String name) {
     return name == null ? null : name.replace("\"", "\"\"");
   }
+
+  /**
+   * Strips outer single-quote or dollar-quote delimiters from a stage name, unescapes doubled
+   * single quotes, and ensures the result starts with {@code @}.
+   */
+  static String normalizeStageNameForRef(String stageName) {
+    if (stageName == null) {
+      return null;
+    }
+    String normalized = stageName;
+    if (isDollarQuotedStageRef(normalized)) {
+      normalized = normalized.substring(2, normalized.length() - 2);
+    } else if (isSingleQuotedStageRef(normalized)) {
+      normalized = unescapeSingleQuotedContent(normalized.substring(1, normalized.length() - 1));
+    }
+    if (!normalized.startsWith("@")) {
+      normalized = "@" + normalized;
+    }
+    return normalized;
+  }
+
+  /** Builds an unquoted {@code @stage/file} reference for GET commands. */
+  static String buildStageFileRef(String stageName, String fileName) {
+    StringBuilder ref = new StringBuilder(normalizeStageNameForRef(stageName));
+    ref.append("/");
+    if (fileName != null && fileName.startsWith("/")) {
+      fileName = fileName.substring(1);
+    }
+    ref.append(fileName);
+    return ref.toString();
+  }
+
+  private static String unescapeSingleQuotedContent(String content) {
+    return content.replace("''", "'");
+  }
+
+  private static boolean containsUnescapedSingleQuote(String content) {
+    for (int i = 0; i < content.length(); i++) {
+      if (content.charAt(i) == '\'') {
+        if (i + 1 < content.length() && content.charAt(i + 1) == '\'') {
+          i++;
+        } else {
+          return true;
+        }
+      }
+    }
+    return false;
+  }
+
+  private static boolean isSingleQuotedStageRef(String stageRef) {
+    if (stageRef == null || stageRef.length() < 2) {
+      return false;
+    }
+    if (!stageRef.startsWith("'") || !stageRef.endsWith("'")) {
+      return false;
+    }
+    return !containsUnescapedSingleQuote(stageRef.substring(1, stageRef.length() - 1));
+  }
+
+  private static boolean isDollarQuotedStageRef(String stageRef) {
+    return stageRef != null
+        && stageRef.length() >= 4
+        && stageRef.startsWith("$$")
+        && stageRef.endsWith("$$");
+  }
+
+  /**
+   * Returns true if a stage reference contains characters that require single-quote wrapping in
+   * PUT/GET SQL. Safe characters are ASCII alphanumerics and {@code _$./@~%"-:} which are valid
+   * unquoted in Snowflake file-operation SQL. Anything else (spaces, non-ASCII/unicode, backslash,
+   * wildcard chars) triggers quoting.
+   */
+  static boolean needsStageQuoting(String stageRef) {
+    if (stageRef == null) {
+      return false;
+    }
+    if (isSingleQuotedStageRef(stageRef) || isDollarQuotedStageRef(stageRef)) {
+      return false;
+    }
+    for (int i = 0; i < stageRef.length(); i++) {
+      char c = stageRef.charAt(i);
+      if ((c >= 'A' && c <= 'Z')
+          || (c >= 'a' && c <= 'z')
+          || (c >= '0' && c <= '9')
+          || "_$./@~%\"-:".indexOf(c) >= 0) {
+        continue;
+      }
+      return true;
+    }
+    return false;
+  }
+
+  /**
+   * Wraps a stage reference in single quotes if it contains characters that require quoting (e.g.
+   * non-ASCII/unicode characters in schema or stage names). Any embedded single quotes are doubled.
+   * Already-quoted references (single-quote or dollar-quote) are returned unchanged.
+   */
+  static String quoteStageRefIfNeeded(String stageRef) {
+    if (stageRef == null || !needsStageQuoting(stageRef)) {
+      return stageRef;
+    }
+    return "'" + stageRef.replace("'", "''") + "'";
+  }
 }
diff --git a/src/test/java/net/snowflake/client/internal/api/implementation/connection/StageQuotingTest.java b/src/test/java/net/snowflake/client/internal/api/implementation/connection/StageQuotingTest.java
@@ -0,0 +1,105 @@
+package net.snowflake.client.internal.api.implementation.connection;
+
+import static net.snowflake.client.internal.api.implementation.connection.SnowflakeConnectionImpl.buildStageFileRef;
+import static net.snowflake.client.internal.api.implementation.connection.SnowflakeConnectionImpl.needsStageQuoting;
+import static net.snowflake.client.internal.api.implementation.connection.SnowflakeConnectionImpl.normalizeStageNameForRef;
+import static net.snowflake.client.internal.api.implementation.connection.SnowflakeConnectionImpl.quoteStageRefIfNeeded;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import org.junit.jupiter.api.Test;
+
+/** Unit tests for stage-reference quoting helpers in {@link SnowflakeConnectionImpl}. */
+public class StageQuotingTest {
+
+  @Test
+  public void needsStageQuotingReturnsFalseForAsciiRefs() {
+    assertFalse(needsStageQuoting("@stage"));
+    assertFalse(needsStageQuoting("@stage/dir/file.csv"));
+    assertFalse(needsStageQuoting("@~/dir/file.csv"));
+    assertFalse(needsStageQuoting("@\"DB\".\"SCHEMA\".\"STAGE\""));
+    assertFalse(needsStageQuoting("@db.schema.%table/dir"));
+    assertFalse(needsStageQuoting(null));
+  }
+
+  @Test
+  public void needsStageQuotingReturnsTrueForNonAscii() {
+    assertTrue(needsStageQuoting("@\"DB\".\"日本語\".STAGE"));
+    assertTrue(needsStageQuoting("@\"DQ\".\"自動化専用_変更禁止\".SNOWPARK_TEMP_STAGE_XYZ"));
+    assertTrue(needsStageQuoting("@\"café\".STAGE"));
+  }
+
+  @Test
+  public void needsStageQuotingReturnsTrueForSpacesAndParens() {
+    assertTrue(needsStageQuoting("@\"my stage\"/path/file.csv"));
+    assertTrue(needsStageQuoting("@\"ice cream (nice)\"/hello.txt"));
+  }
+
+  @Test
+  public void needsStageQuotingReturnsFalseForAlreadyQuoted() {
+    assertFalse(needsStageQuoting("'@\"DB\".\"日本語\".STAGE'"));
+    assertFalse(needsStageQuoting("$$@\"DB\".\"日本語\".STAGE$$"));
+  }
+
+  @Test
+  public void needsStageQuotingReturnsTrueForMalformedQuotedRef() {
+    assertTrue(needsStageQuoting("'a'b'c'"));
+  }
+
+  @Test
+  public void normalizeStageNameForRefStripsOuterQuotes() {
+    assertEquals("@stage", normalizeStageNameForRef("stage"));
+    assertEquals("@stage", normalizeStageNameForRef("@stage"));
+    assertEquals("@%\"ice cream (nice)\"", normalizeStageNameForRef("'@%\"ice cream (nice)\"'"));
+    assertEquals("@%\"ice cream (nice)\"", normalizeStageNameForRef("$$@%\"ice cream (nice)\"$$"));
+    assertEquals("@stage_O'Brien", normalizeStageNameForRef("'@stage_O''Brien'"));
+  }
+
+  @Test
+  public void buildStageFileRefHandlesPreQuotedStageName() {
+    assertEquals("@stage/file.csv", buildStageFileRef("@stage", "file.csv"));
+    assertEquals("@stage/file.csv", buildStageFileRef("@stage", "/file.csv"));
+    assertEquals(
+        "@%\"ice cream (nice)\"/hello.txt",
+        buildStageFileRef("'@%\"ice cream (nice)\"'", "hello.txt"));
+    assertEquals(
+        "'@%\"ice cream (nice)\"/hello.txt'",
+        quoteStageRefIfNeeded(buildStageFileRef("'@%\"ice cream (nice)\"'", "hello.txt")));
+  }
+
+  @Test
+  public void quoteStageRefIfNeededQuotesNonAscii() {
+    assertEquals("'@\"DB\".\"日本語\".STAGE'", quoteStageRefIfNeeded("@\"DB\".\"日本語\".STAGE"));
+    assertEquals(
+        "'@\"DQ\".\"自動化専用_変更禁止\".SNOWPARK_TEMP_STAGE_XYZ/file.txt'",
+        quoteStageRefIfNeeded("@\"DQ\".\"自動化専用_変更禁止\".SNOWPARK_TEMP_STAGE_XYZ/file.txt"));
+  }
+
+  @Test
+  public void quoteStageRefIfNeededLeavesAsciiUnchanged() {
+    assertEquals("@stage", quoteStageRefIfNeeded("@stage"));
+    assertEquals("@stage/dir/file.csv", quoteStageRefIfNeeded("@stage/dir/file.csv"));
+    assertEquals("@\"DB\".\"SCHEMA\".STAGE", quoteStageRefIfNeeded("@\"DB\".\"SCHEMA\".STAGE"));
+  }
+
+  @Test
+  public void quoteStageRefIfNeededLeavesAlreadyQuotedUnchanged() {
+    String singleQuoted = "'@\"DB\".\"日本語\".STAGE'";
+    assertEquals(singleQuoted, quoteStageRefIfNeeded(singleQuoted));
+    String dollarQuoted = "$$@\"DB\".\"日本語\".STAGE$$";
+    assertEquals(dollarQuoted, quoteStageRefIfNeeded(dollarQuoted));
+  }
+
+  @Test
+  public void quoteStageRefIfNeededEscapesEmbeddedSingleQuotes() {
+    assertEquals("'@stage_O''Brien'", quoteStageRefIfNeeded("@stage_O'Brien"));
+    assertEquals("'@\"sch''ema\".STAGE'", quoteStageRefIfNeeded("@\"sch'ema\".STAGE"));
+  }
+
+  @Test
+  public void quoteStageRefIfNeededHandlesNull() {
+    assertNull(quoteStageRefIfNeeded(null));
+  }
+}
diff --git a/src/test/java/net/snowflake/client/internal/jdbc/StreamLatestIT.java b/src/test/java/net/snowflake/client/internal/jdbc/StreamLatestIT.java
@@ -359,4 +359,50 @@ private static boolean putFile(
             "PUT file://%s %s AUTO_COMPRESS=%s",
             localFileName, stageDest, String.valueOf(autoCompress).toUpperCase()));
   }
+
+  /** SNOW-3713887: uploadStream/downloadStream with unicode schema in stage name */
+  @Test
+  public void testUploadDownloadStreamWithUnicodeSchemaStage() throws Throwable {
+    String unicodeSchema = "\"日本語テスト_" + UUID.randomUUID().toString().substring(0, 5) + "\"";
+    String stageName = "test_stream_stage_" + UUID.randomUUID().toString().replaceAll("-", "");
+
+    try (Connection connection = getConnection();
+        Statement statement = connection.createStatement()) {
+      String database = connection.getCatalog();
+      String qualifiedStage = database + "." + unicodeSchema + "." + stageName;
+
+      try {
+        statement.execute("CREATE SCHEMA IF NOT EXISTS " + database + "." + unicodeSchema);
+        statement.execute("CREATE TEMPORARY STAGE " + qualifiedStage);
+
+        String content = "hello from unicode schema";
+        String fileName = "unicode_test.txt";
+
+        connection
+            .unwrap(SnowflakeConnection.class)
+            .uploadStream(
+                "@" + qualifiedStage,
+                fileName,
+                new ByteArrayInputStream(content.getBytes(StandardCharsets.UTF_8)),
+                UploadStreamConfig.builder().setCompressData(false).build());
+
+        try (InputStream in =
+            connection
+                .unwrap(SnowflakeConnection.class)
+                .downloadStream(
+                    "@" + qualifiedStage,
+                    "/" + fileName,
+                    DownloadStreamConfig.builder().setDecompress(false).build())) {
+          String downloaded =
+              new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8))
+                  .lines()
+                  .collect(Collectors.joining("\n"));
+          assertEquals(content, downloaded);
+        }
+      } finally {
+        statement.execute("DROP STAGE IF EXISTS " + qualifiedStage);
+        statement.execute("DROP SCHEMA IF EXISTS " + database + "." + unicodeSchema);
+      }
+    }
+  }
 }
