Skip to content

Add AnyBinary: one logical type for any binary encoding#9

Merged
emilk merged 1 commit into
mainfrom
emilk/any-binary
Jun 9, 2026
Merged

Add AnyBinary: one logical type for any binary encoding#9
emilk merged 1 commit into
mainfrom
emilk/any-binary

Conversation

@emilk

@emilk emilk commented Jun 9, 2026

Copy link
Copy Markdown
Member

The byte-string sibling of AnyList: a quiver-only logical type that accepts any of arrow's binary encodings and reads them uniformly as &[u8].

Logical type Accepts Element value
AnyBinary Binary, LargeBinary, BinaryView, FixedSizeBinary (any size) &[u8]
let column = Column::<AnyBinary>::try_from(array)?; // Binary / LargeBinary / BinaryView / FixedSizeBinary
assert_eq!(column.value(0), b"abc");

Details

  • Backed by an enum AnyTypedBinary over the four arrow binary arrays; downcast dispatches on array.data_type() and reads every element as &[u8].
  • FixedSizeBinary is included (mirroring AnyList including FixedSizeList): at the arrow level FixedSizeBinaryArray::value(i) already returns &[u8] (length N), so it fits the uniform &[u8] element type — only quiver's dedicated FixedSizeBinary<N> wrapper exposes &[u8; N].
  • Parse-only: implements LogicalType (so try_from/reading/column[i] work) but not ConcreteType — no single arrow datatype, so no from_values/Default/schema. Build a concrete encoding such as Column<Binary>.
  • Reads stay O(1)/zero-copy; nullability via Option<AnyBinary> (top-level) like any other column.

Verification

  • cargo clippy --all-features --all-targets clean, cargo fmt --all applied
  • cargo test --all-features green, incl. a new any_binary_columns test: all four encodings read uniformly, RefType indexing, non-binary rejected, null handling (Option<AnyBinary> + non-nullable rejection, including FixedSizeBinary)
  • cargo doc --document-private-items -D warnings clean
  • README: added to the supported-types table and the AnyList section

🤖 Generated with Claude Code

Mirrors `AnyList`: `Column<AnyBinary>` accepts any of arrow's byte-string
encodings — `Binary`, `LargeBinary`, `BinaryView`, or `FixedSizeBinary` (any
size) — and reads them all uniformly as `&[u8]` (a `FixedSizeBinary<N>`'s
fixed-width elements are seen as plain `&[u8]` slices, not `&[u8; N]`). Like
`AnyList` it is parse-only: implements `LogicalType` but not `ConcreteType`, so
no `from_values`/`Default`/schema — build a concrete encoding such as
`Column<Binary>`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@emilk emilk merged commit e57d395 into main Jun 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant