Skip to content

Follow-up: document the icebird-backed absent-column contract deferred from PR #740 #778

Description

@philcunliffe

One deferred item from PR #740 ("The union's absent-column contract is undefined-or-throws, not null"). It is not a defect in that PR: the maintainer explicitly descoped it (option 2 in the triage thread) after the icebird-facing text had been described wrongly five times in a row, and directed that it land later in its own change. This issue exists so that direction does not get lost.

Backlink: PR #740, triage thread (stuck report of 2026-08-13 and the maintainer decision comment of 2026-08-14). Line numbers are as of head f073338.

Document the icebird-backed absent-column contract, with SQL-surface tests pinning it

PR #740 corrected the parquet-union contract (undefined-or-throws) and verified it. The equivalent contract for the icebird-backed flagship dataset (ai_gateway_messages) is deliberately not documented anywhere on master after that PR merges; three places now carry neutral placeholder wording that defers to "the read path":

  • hypaware-core/plugins-workspace/ai-gateway/src/dataset.js:166-169 - the withSchemaColumns block comment: "a column an old partition physically lacks stays addressable, and the exact value a read of it yields depends on the read path (LLP 0015#multi-partition-union)."
  • hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:25-28 - same deferral for the v7 nullable capture columns.
  • llp/0015-query-and-datasets.spec.md:102-105 - the wrapper clause ends "the exact value a read of it then yields depends on the read path and is not settled here."

What the triage measured (to be re-verified and pinned, not trusted)

On a real icebird-backed ai_gateway_messages fixture with drifted partitions:

  • SELECT git_remote FROM t returns null (rendered, not dropped): squirreling routes any single-column scan through scanColumn when the source offers one (execute.js:284 gates only on plan.hints.columns?.length === 1, no aggregate required), and withSchemaColumns normalizes holes to null on exactly that path (dataset.js:216-224 at PR head numbering).
  • SELECT date, git_remote FROM t (multi-column bare projection) reads undefined, key dropped by JSON.stringify.
  • Nothing throws, because icebird supplies a resolving cell for a declared-but-absent column; the throwing half of LLP 0015 belongs to parquet-backed partitions, which omit the cell entirely.

Five successive descriptions of this mechanism were each measured false during PR #740's review, so the doc change must land together with tests, not before them.

The tests the maintainer asked for

executeSql + collect SQL-surface tests on the icebird fixture, so the claim is pinned where it was previously untested. test/core/ai-gateway-dataset.test.js currently pins only the raw scan() rows (lines 257-294) and the scanColumn chunks (lines 296-327); no test runs a full SELECT through executeSql + collect over an icebird-backed drifted partition. At minimum: single-column bare projection (the scanColumn fast path), multi-column bare projection (the resolveable/collect() path), a projection with a non-identifier sibling, and a WHERE/aggregate on the absent column, each asserting the exact value (null versus undefined versus throw) rather than a ?? null tolerant form.

Doc side

Whether the icebird contract belongs as an extension in LLP 0015, a new LLP, or only in the two code comments is the implementer's call, subject to the LLP immutability rules. The neutral placeholder wording in the three locations above should then be replaced by the pinned claim.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    neutral:fixDelegate this issue to neutral for an autonomous fix attempt (reproduce -> fix -> PR)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions