Skip to content

[WIp] ALP encoder/decoder support#9372

Open
sdf-jkl wants to merge 44 commits into
apache:mainfrom
sdf-jkl:alp
Open

[WIp] ALP encoder/decoder support#9372
sdf-jkl wants to merge 44 commits into
apache:mainfrom
sdf-jkl:alp

Conversation

@sdf-jkl

@sdf-jkl sdf-jkl commented Feb 7, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

check issue

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the parquet Changes to the parquet crate label Feb 7, 2026
@alamb

alamb commented Feb 8, 2026

Copy link
Copy Markdown
Contributor

amaaazing

@sdf-jkl

sdf-jkl commented Feb 20, 2026

Copy link
Copy Markdown
Contributor Author

@alamb I worked with codex on replicating the c++ implementation reviewing commit by commit.

I'll do one final read myself, but this should be ready for initial review.

@alamb

alamb commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

Thank you -- I will put this on my short list to reivew I was out last week

@devanbenz devanbenz left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments

Comment thread parquet/src/encodings/decoding/alp.rs Outdated
Comment thread parquet/src/encodings/decoding/alp.rs Outdated
Comment thread parquet/src/encodings/decoding/alp.rs Outdated
@sdf-jkl

sdf-jkl commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

@devanbenz Sorry for taking so long to address your comments. I'll make sure to work on it this week!

@alamb

alamb commented Mar 11, 2026

Copy link
Copy Markdown
Contributor

The final spec is

@devanbenz devanbenz left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment about header regarding publication of spec.

Comment thread parquet/src/encodings/decoding/alp.rs Outdated
@sdf-jkl

sdf-jkl commented Mar 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @devanbenz, I've finally addressed your reviews!

@devanbenz

Copy link
Copy Markdown

Thanks @devanbenz, I finally addressed your reviews!

Please cargo fmt the code 🫡

@sdf-jkl

sdf-jkl commented Mar 12, 2026

Copy link
Copy Markdown
Contributor Author

I swear I did 😿
Had to run

cargo fmt -p parquet --  --config skip_children=true `find ./parquet -name "*.rs" \! -name format.rs`

5e5c8d2

sdf-jkl and others added 3 commits May 8, 2026 22:29
Refactor: Pull common header structures into common module
@alamb alamb changed the title [WIp] Alp encoding support [WIp] ALP encoder/decoder support May 9, 2026
@alamb

alamb commented May 9, 2026

Copy link
Copy Markdown
Contributor

here is another small proposed improvement

sdf-jkl and others added 2 commits May 9, 2026 11:32
alamb added a commit that referenced this pull request May 14, 2026
# Which issue does this PR close?

- Related to #9372


# Rationale for this change

Similarly to #9948. I ran into
BitWriter as part of reviewing code from @sdf-jkl and wanted to document
my findings (so I didn't have to re-read the code each time)
- #9372



# What changes are included in this PR?

Add docs 

# Are these changes tested?

By CI
# Are there any user-facing changes?

No -- this is docs to an internal structure
alamb added a commit that referenced this pull request May 14, 2026
# Which issue does this PR close?

- Related to #9372

# Rationale for this change

While reviewing the ALP implementation from @sdf-jkl , I ran into this
struct which I haven't really used before.
- #9372

Now that I have read it, I wanted to capture that information as doc
comments (for my future self and hopefully for others)

# What changes are included in this PR?

Add documentation comments to `BitReader`

# Are these changes tested?

Just docs, 

# Are there any user-facing changes?
Just docs on an internal struct,
Rich-T-kid pushed a commit to Rich-T-kid/arrow-rs that referenced this pull request Jun 2, 2026
# Which issue does this PR close?

- Related to apache#9372


# Rationale for this change

Similarly to apache#9948. I ran into
BitWriter as part of reviewing code from @sdf-jkl and wanted to document
my findings (so I didn't have to re-read the code each time)
- apache#9372



# What changes are included in this PR?

Add docs 

# Are these changes tested?

By CI
# Are there any user-facing changes?

No -- this is docs to an internal structure
Rich-T-kid pushed a commit to Rich-T-kid/arrow-rs that referenced this pull request Jun 2, 2026
# Which issue does this PR close?

- Related to apache#9372

# Rationale for this change

While reviewing the ALP implementation from @sdf-jkl , I ran into this
struct which I haven't really used before.
- apache#9372

Now that I have read it, I wanted to capture that information as doc
comments (for my future self and hopefully for others)

# What changes are included in this PR?

Add documentation comments to `BitReader`

# Are these changes tested?

Just docs, 

# Are there any user-facing changes?
Just docs on an internal struct,
@sdf-jkl

sdf-jkl commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

@alamb @devanbenz 🤓

@sdf-jkl sdf-jkl marked this pull request as ready for review June 9, 2026 16:43
@alamb

alamb commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Thanks @sdf-jkl -- I will try and find tiume to review this (probably not until the weekend though as I have to sort out the next object store release before

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Parquet] Prototype ALP encoding

3 participants