Skip to content

refactor(parquet): introduce string_offset_scan_state for preprocess_string_offsets_kernel - #23495

Open
vyasr wants to merge 3 commits into
rapidsai:mainfrom
vyasr:parquet-substruct-redesign/pr4-progress-string-offset
Open

refactor(parquet): introduce string_offset_scan_state for preprocess_string_offsets_kernel#23495
vyasr wants to merge 3 commits into
rapidsai:mainfrom
vyasr:parquet-substruct-redesign/pr4-progress-string-offset

Conversation

@vyasr

@vyasr vyasr commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Description

This PR builds on #23479 to continue narrowing the shared state used by Parquet preprocessing kernels. It extracts page_decode_progress_state from page_state_s, grouping the decode-position/progress fields that are needed by string-offset preprocessing.

The primary addition is string_offset_scan_state, composed from page_decode_setup_state, page_decode_stream_state, and page_decode_progress_state. preprocess_string_offsets_kernel now uses this smaller composed state instead of the full page_state_s, which drops unrelated output-conversion, nesting, and level-scratch fields from that kernel's shared-memory state. This follows the same pattern as #23479: pull one coherent sub-state out of page_state_s, migrate the target kernel to the minimal composed state it actually needs, and leave the remaining full decode state for a later PR.

A fresh cuobjdump -res-usage comparison against upstream/main shows preprocess_string_offsets_kernel saves 512 bytes of shared memory on every generated architecture, with no register-count changes:

Arch shmem before shmem after delta regs before regs after
sm_70 2048 1536 -512 40 40
sm_75 2048 1536 -512 64 64
sm_80 2048 1536 -512 37 37
sm_86 2048 1536 -512 38 38
sm_90a 3072 2560 -512 32 32
sm_100f 3072 2560 -512 32 32
sm_120a 3072 2560 -512 40 40
sm_120 3072 2560 -512 40 40

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

…rocess_string_offsets_kernel via string_offset_scan_state
@vyasr
vyasr requested a review from a team as a code owner July 31, 2026 04:52
@vyasr
vyasr requested review from bdice and mythrocks July 31, 2026 04:52
@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Jul 31, 2026
@vyasr vyasr added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jul 31, 2026
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 75716962-9d71-4041-b107-d3dc467244bf

📥 Commits

Reviewing files that changed from the base of the PR and between 43ca9a2 and 03f9c78.

📒 Files selected for processing (1)
  • cpp/src/io/parquet/page_state_composed.cuh
🚧 Files skipped from review as they are similar to previous changes (1)
  • cpp/src/io/parquet/page_state_composed.cuh

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Improved Parquet decoding reliability for fixed-width, string, dictionary-encoded, boolean, delta-encoded, and nested data.
    • Improved handling of page boundaries, row limits, validity information, and list data.
    • Improved string offset processing and page containment checks for more consistent results.
    • Improved source-position tracking and value-count handling during dictionary and delta decoding.

Walkthrough

Parquet decoding now stores cumulative counters and row-bound state in page_state_s::progress. Page-boundary helpers use page state directly. Fixed-width, page, delta, and string decoders were updated. String offset preprocessing now uses a composed state.

Changes

Parquet progress-state migration

Layer / File(s) Summary
Progress state and page-boundary contracts
cpp/src/io/parquet/page_decode.cuh
Adds page_decode_progress_state, nests progress in page_state_s, updates boundary helpers, and initializes progress fields.
Fixed-width decoding and preprocessing
cpp/src/io/parquet/decode_fixed.cu, cpp/src/io/parquet/decode_preprocess.cu
Routes validity, row, value, leaf, dictionary, and row-bound counters through s->progress.
Page and delta decoder progress
cpp/src/io/parquet/page_data.cu, cpp/src/io/parquet/page_delta_decode.cu
Updates page and delta decoders to use nested source-position, dictionary-position, and input counters.
String offset scan state
cpp/src/io/parquet/page_state_composed.cuh, cpp/src/io/parquet/page_string_decode.cu
Adds string_offset_scan_state and updates string offset readers and page-boundary calls to use composed state.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

  • rapidsai/cudf#23241: Refactors the same Parquet page-boundary helpers and preprocessing and string-decoding call sites.
  • rapidsai/cudf#23374: Overlaps in the Parquet page-state and decoder paths modified by this change.
  • rapidsai/cudf#23471: Refactors page_state_s access across the same Parquet decoder kernels.

Suggested reviewers: bdice, mythrocks

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: introducing string_offset_scan_state for Parquet string-offset preprocessing.
Description check ✅ Passed The description accurately explains the state refactoring, kernel change, and shared-memory reduction.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/src/io/parquet/page_decode.cuh`:
- Around line 204-207: Update the Doxygen blocks for is_bounds_page and the
other affected helper declarations to document the actual s parameter, replacing
the obsolete page and chunk_start_row entries. Ensure every documented parameter
matches its corresponding signature, including start_row, num_rows, and
has_repetition.

In `@cpp/src/io/parquet/page_state_composed.cuh`:
- Around line 47-50: Update the static_assert involving string_offset_scan_state
to compare its size directly against sizeof(page_state_s), removing the
additional page_decode_* size terms so the assertion detects shared-memory size
regressions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aeb3f98f-cc50-4e63-8a04-2c5908743ce3

📥 Commits

Reviewing files that changed from the base of the PR and between 7f58752 and 4f6848d.

📒 Files selected for processing (7)
  • cpp/src/io/parquet/decode_fixed.cu
  • cpp/src/io/parquet/decode_preprocess.cu
  • cpp/src/io/parquet/page_data.cu
  • cpp/src/io/parquet/page_decode.cuh
  • cpp/src/io/parquet/page_delta_decode.cu
  • cpp/src/io/parquet/page_state_composed.cuh
  • cpp/src/io/parquet/page_string_decode.cu

Comment thread cpp/src/io/parquet/page_decode.cuh
Comment thread cpp/src/io/parquet/page_state_composed.cuh Outdated
CUDF_PARQUET_PAGE_STATE_ERROR_METHODS
};
static_assert(sizeof(string_offset_scan_state) < sizeof(page_state_s),
"string_offset_scan_state did not shrink after removing output conversion state");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to be left in?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be removed, but since CI already passed here unless another reviewer requests changes I'll just leave it for now and clean it up in a follow-up PR. We'll be either renaming/removing page_state_s so this'll have to change then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants