Add logical type inspection for raw Parquet VARIANT values

[abigalekim]

Description

Implements feature mentioned in #23183. Adds an experimental libcudf operation, cudf::io::parquet::experimental::get_variant_type_id, that takes a raw Variant list column and returns an int32 column of logical type identifiers. This PR also introduces a new variant_logical_type enum covering all Variant categories.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b80a0fd1-1fe4-40ca-a290-38732a8b3916

📥 Commits

Reviewing files that changed from the base of the PR and between e3930a4 and 512ffdc.

📒 Files selected for processing (3)

• cpp/include/cudf/io/experimental/variant_spec.hpp
• cpp/src/io/parquet/experimental/variant_extract.cu
• cpp/tests/io/experimental/variant_extract_test.cpp

🚧 Files skipped from review as they are similar to previous changes (3)

• cpp/include/cudf/io/experimental/variant_spec.hpp
• cpp/src/io/parquet/experimental/variant_extract.cu
• cpp/tests/io/experimental/variant_extract_test.cpp

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added support for identifying the logical type of each VARIANT value, including objects, arrays, strings, numbers, dates, timestamps, binary values, UUIDs, and nulls.
  • Results are returned as integer type identifiers while preserving input nulls and explicitly representing encoded null values.
  • Added handling for empty, sliced, mixed, malformed, and unsupported VARIANT data.

• Tests

  • Added comprehensive coverage across supported logical types, null handling, malformed inputs, and large datasets.

Walkthrough

Adds the public variant_logical_type enum and get_variant_type_id API. CUDA code decodes VARIANT headers into nullable INT32 type IDs. Tests cover mappings, malformed values, nulls, empty inputs, slicing, mixed types, and large columns.

Changes

VARIANT type-ID extraction

Layer / File(s)	Summary
Logical type contract cpp/include/cudf/io/experimental/variant_spec.hpp, cpp/include/cudf/io/experimental/variant.hpp	Defines variant_logical_type and declares get_variant_type_id with null and invalid-input behavior.
Type-ID classification and output cpp/src/io/parquet/experimental/variant_extract.cu	Classifies VARIANT headers on the device and returns nullable INT32 output.
Classification and edge-case coverage cpp/tests/io/experimental/variant_extract_test.cpp	Tests fixture mappings, malformed and unknown values, encoded nulls, empty inputs, slicing, mixed types, and large columns.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

• [FEA] Add logical type inspection for raw Parquet VARIANT values #23183 — Covers the variant_logical_type and get_variant_type_id feature.
• Support logical type inspection for Parquet VARIANT values #23182 — Covers the corresponding libcudf logical-type inspection operation.
• Add Java bindings for Parquet VARIANT logical type inspection #23184 — Covers the API proposed for Java bindings.
• [FEA][Story] Support ingest of Parquet VARIANT columns #22312 — Covers related VARIANT logical-type decoding APIs.

Possibly related PRs

• rapidsai/cudf#23036 — Shares VARIANT definitions, extraction code, and related tests.
• rapidsai/cudf#23400 — Extends the Parquet VARIANT APIs and shares extraction test infrastructure.

Suggested labels: cuIO, tests, improvement

Suggested reviewers: mattgara, vuule, davidwendt

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding logical type inspection for raw Parquet VARIANT values.
Description check	✅ Passed	The description accurately explains the new API, enum, input and output types, issue reference, tests, and documentation updates.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

cpp/include/cudf/io/experimental/variant_spec.hpp (1)

56-71: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assign explicit values to variant_logical_type.

The enumerators of variant_logical_type rely on implicit sequential values. get_variant_type_id casts these values to int32_t and returns them as output data. If a new logical type is inserted in the middle of this list later, every subsequent numeric ID shifts silently. This can break downstream consumers that persist or compare these IDs.

Assign each enumerator an explicit value now, while the API is new, to fix the wire contract.

♻️ Proposed fix to pin enumerator values

 enum class variant_logical_type : uint8_t {
-  object,
-  array,
-  null_value,
-  boolean,
-  long_value,
-  string,
-  double_value,
-  decimal,
-  date,
-  timestamp,
-  timestamp_ntz,
-  float_value,
-  binary,
-  uuid
+  object        = 0,
+  array         = 1,
+  null_value    = 2,
+  boolean       = 3,
+  long_value    = 4,
+  string        = 5,
+  double_value  = 6,
+  decimal       = 7,
+  date          = 8,
+  timestamp     = 9,
+  timestamp_ntz = 10,
+  float_value   = 11,
+  binary        = 12,
+  uuid          = 13
 };

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/include/cudf/io/experimental/variant_spec.hpp` around lines 56 - 71,
Update the variant_logical_type enum so every enumerator has an explicit numeric
value, preserving its current sequential IDs and establishing a stable wire
contract for get_variant_type_id. Use the existing declaration order and assign
values starting at zero through uuid; do not change the enum members or their
ordering.

cpp/src/io/parquet/experimental/variant_extract.cu (1)

790-817: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the repeated value-span construction into a helper.

Lines 803-806 repeat the same offset/span construction already used in cast_variant_primitive_kernel (lines 632-636) and cast_variant_string_fn::operator() (lines 670-674). This is now the third copy of the same three-line idiom.

Extract a small __device__ helper that returns the value device_span<uint8_t const> for a row, and call it from all three sites. This reduces duplication and keeps future changes to the span-lookup logic in one place.

♻️ Proposed helper extraction

__device__ inline device_span<uint8_t const> value_span_at(
  cudf::lists_column_device_view const& values, size_type row)
{
  auto const val_begin = values.offset_at(row);
  auto const val_end   = values.offset_at(row + 1);
  return {values.child().data<uint8_t>() + val_begin,
          static_cast<std::size_t>(val_end - val_begin)};
}

     auto const val_begin = values.offset_at(row);
     auto const val_end   = values.offset_at(row + 1);
-    device_span<uint8_t const> const val{values.child().data<uint8_t>() + val_begin,
-                                         static_cast<std::size_t>(val_end - val_begin)};
+    auto const val = value_span_at(values, row);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/src/io/parquet/experimental/variant_extract.cu` around lines 790 - 817,
Extract the repeated value offset/span construction into a shared __device__
helper, such as value_span_at, accepting the lists_column_device_view and row
and returning device_span<uint8_t const>. Replace the duplicated three-line
logic in get_variant_type_id_kernel, cast_variant_primitive_kernel, and
cast_variant_string_fn::operator() with calls to this helper, preserving the
existing span contents and behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tests/io/experimental/variant_extract_test.cpp`:
- Around line 1621-1657: Increase num_rows in
GetVariantTypeIdTest.LargeMultiRowColumn from 128 to a value above the kernel
block size, such as 600, so get_variant_type_id exercises the multi-block
grid-stride path while preserving the existing cycling type coverage and
expected_ids generation.

---

Nitpick comments:
In `@cpp/include/cudf/io/experimental/variant_spec.hpp`:
- Around line 56-71: Update the variant_logical_type enum so every enumerator
has an explicit numeric value, preserving its current sequential IDs and
establishing a stable wire contract for get_variant_type_id. Use the existing
declaration order and assign values starting at zero through uuid; do not change
the enum members or their ordering.

In `@cpp/src/io/parquet/experimental/variant_extract.cu`:
- Around line 790-817: Extract the repeated value offset/span construction into
a shared __device__ helper, such as value_span_at, accepting the
lists_column_device_view and row and returning device_span<uint8_t const>.
Replace the duplicated three-line logic in get_variant_type_id_kernel,
cast_variant_primitive_kernel, and cast_variant_string_fn::operator() with calls
to this helper, preserving the existing span contents and behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f1423210-95f8-4ad2-ab54-2d2cafd21e53

📥 Commits

Reviewing files that changed from the base of the PR and between 7f58752 and 7bbdfb7.

📒 Files selected for processing (4)

• cpp/include/cudf/io/experimental/variant.hpp
• cpp/include/cudf/io/experimental/variant_spec.hpp
• cpp/src/io/parquet/experimental/variant_extract.cu
• cpp/tests/io/experimental/variant_extract_test.cpp

[abigalekim]

/ok to test 2dc6168