Skip to content

Adaptive Lossless Floating Point - #161

Open
platypii wants to merge 1 commit into
masterfrom
alp
Open

Adaptive Lossless Floating Point#161
platypii wants to merge 1 commit into
masterfrom
alp

Conversation

@platypii

@platypii platypii commented May 6, 2026

Copy link
Copy Markdown
Collaborator

Adds an ALP (Adaptive Lossless floating-Point) decoder for FLOAT and DOUBLE columns, per the parquet-format spec: https://github.com/apache/parquet-format/blob/master/AlpEncoding.md

  • Page layout: 7 byte header, uint32 offset array, self-describing vectors (AlpInfo, ForInfo, packed values, exceptions).
  • Decode is encoded * 10^factor * 10^-exponent with correctly rounded constants. FLOAT decodes in float32 arithmetic to match the reference implementation.
  • Frame of reference is applied with wrapping int32/int64 arithmetic. DOUBLE uses a Number fast path when bit width <= 32 and FOR is a safe integer, BigInt otherwise.
  • Exception values are copied as raw bytes so NaN payloads are preserved.

Tests:

  • Unit tests build pages from the spec layout, including the worked example from the spec.
  • alp_extended.zstd.parquet from parquet-testing is added to test files. All six ALP columns are compared bitwise against the PLAIN columns (9032 rows, vector sizes 32/1024/4096, NaN payloads, Inf, -0, subnormals, nulls, 64 bit FOR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant