Skip to content

feat: add options support to Store for format-specific storage configuration - #19

Merged
sfishel18 merged 1 commit into
wherobots:mainfrom
james-willis:feat/store-options
Mar 3, 2026
Merged

feat: add options support to Store for format-specific storage configuration#19
sfishel18 merged 1 commit into
wherobots:mainfrom
james-willis:feat/store-options

Conversation

@james-willis

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new Store.forDownload(StorageFormat format, Map<String, String> options) factory method, allowing users to pass format-specific storage options (e.g. ignoreNullFields, header, delimiter) that correspond to Spark's OPTIONS (...) clause
  • The options map is serialized as a nested JSON object in the store field of the execute_sql request
  • Existing forDownload() and forDownload(format) methods are fully backward compatible — no changes required for current users

Motivation

Users storing results via wstmt.setStore(Store.forDownload()) had no way to pass configurations like:

CREATE TABLE output_table
USING geojson
OPTIONS (ignoreNullFields 'false')

Now they can:

wstmt.setStore(Store.forDownload(StorageFormat.geojson, Map.of("ignoreNullFields", "false")));

Changes

  • Store.java — Added Map<String, String> options field, new forDownload(format, options) factory method, and getOptions() getter. Options are defensively copied and stored as an unmodifiable map; empty/null options are normalized to null and omitted from JSON serialization.
  • StoreTest.java (new) — 8 tests covering serialization, backward compatibility, empty/null handling, immutability, and end-to-end ExecuteSqlRequest serialization.
  • README.md — Updated Store Options section with the new factory method and usage examples.

…uration

Allow users to pass format-specific options (e.g. ignoreNullFields, header,
delimiter) when using Store.forDownload(), corresponding to Spark's OPTIONS
clause. Adds a new forDownload(format, options) factory method while keeping
the existing API fully backward compatible.
@james-willis
james-willis requested a review from a team as a code owner March 2, 2026 20:00
@james-willis
james-willis requested review from sfishel18 and removed request for a team March 2, 2026 20:00
@james-willis
james-willis marked this pull request as draft March 2, 2026 20:08
@james-willis
james-willis marked this pull request as ready for review March 2, 2026 21:24
@sfishel18
sfishel18 merged commit 61249b9 into wherobots:main Mar 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants