Skip to content

reader(csharp): SELECT result-set Arrow nullable flag false (Thrift) vs true (SEA) — schema-only, data matches #529

Description

@eric-wang-1990

Part of #519 (Thrift vs SEA metadata parity).

Problem

For ordinary (non-metadata) SELECT result sets, the Arrow schema's per-column nullable flag differs between protocols:

columns[<col>] / nullable : thrift=false → sea=true

This is the only difference in every STATEMENT_SELECT, NULL_HANDLING, and most STATEMENT_OTHER SELECT case (e.g. id, database, catalog, col_name, key, count(1)), and it recurs under the COMPRESSION_DISABLED / DIRECT_RESULTS_DISABLED / NO_CLOUDFETCH / PRO_WAREHOUSE configs too. Thrift marks columns non-nullable; SEA marks them nullable.

Importantly: the row data is identical between protocols — this is a schema-metadata-only divergence. Lowest functional risk, but high volume and it means the Arrow schema a consumer sees depends on the protocol.

Expected

Both protocols should report the same nullable flag in the result Arrow schema.

How JDBC does it

JDBC's DatabricksResultSetMetaData defaults every column to Nullable.NULLABLE (isNullable() returns ResultSetMetaData.columnNullable = 1) unless the server explicitly says otherwise — i.e. SEA's nullable=true matches the JDBC default. The Thrift path's nullable=false is the outlier.

Suggested fix

Align the Thrift reader's Arrow-schema nullable flag with SEA (default nullable=true unless the server reports non-nullable), so the result schema is protocol-independent.

Evidence

diff-report-20260615-064733Z.json, suites STATEMENT_SELECT / NULL_HANDLING / STATEMENT_OTHER, where = step[*] / columns[*] / nullable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETPull requests that update .NET codebugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions