[VL] shuffle: TypeAwareCompress(tac) for column-wise data compression like (U)INT64 by guowangy · Pull Request #11894 · apache/gluten

guowangy · 2026-04-09T02:25:55Z

What changes are proposed in this pull request?

Introduces TypeAwareCompress (TAC) — a column-wise compression layer for shuffle that selects
an algorithm based on each buffer's data type, applied per-buffer alongside the existing LZ4/ZSTD
codec path.

For INT64/UINT64 columns the values are often clustered in a small range, making
Frame-of-Reference + Bit-Packing (FFOR) significantly more effective than generic byte-level
compression. TAC exploits this by encoding 8-byte integer buffers with a 4-lane FFOR codec before
the standard codec sees them.

Here is the performance data on TPCH/TPCDS:

	Total Latency	Shuffle Write Size
TPCH-6T	-15%	-32%
TPCDS-6T	-6%	-14%

New files

Path	Purpose
`cpp/core/utils/tac/ffor.hpp`	Header-only 4-lane FFOR codec for `uint64_t`
`cpp/core/utils/tac/FForCodec.{h,cc}`	Arrow-Result wrapper around `ffor.hpp`
`cpp/core/utils/tac/TypeAwareCompressCodec.{h,cc}`	Type dispatch; self-describing wire format (codec ID + element width embedded in header, so decompression needs no type hint)
`cpp/velox/shuffle/VeloxTypeAwareCompress.h`	Maps Velox `TypeKind` → `TacDataType` (`BIGINT` → `kUInt64`)

Shuffle integration

Payload.cc/h: BlockPayload::fromBuffers accepts an optional bufferTypes vector. Per-buffer:
if TypeAwareCompressCodec::support(type) is true, use TAC; otherwise fall back to LZ4/ZSTD.
A new wire marker kTypeAwareBuffer = -3 is added; decompression in readCompressedBuffer is
self-describing. If TAC compressed size ≥ original, falls back to kUncompressedBuffer.
Options.h: adds enableTypeAwareCompress (default false) to LocalPartitionWriterOptions.
VeloxHashShuffleWriter: populates bufferTypes from the schema when TAC is enabled.
GlutenConfig.scala: new config spark.gluten.sql.columnar.shuffle.typeAwareCompress.enabled (default false).
ColumnarShuffleWriter / LocalPartitionWriterJniWrapper: forward the new option to native.

Disabled by default — no behaviour changes for existing deployments.

How was this patch tested?

cpp/core/tests/FForCodecTest.cc covers:

Round-trip correctness for random, all-zero, monotonic, and near-max value patterns
maxCompressedLength boundary checks
Invalid input size rejection

cpp/velox/tests/VeloxShuffleWriterTest.cc: extended to exercise the TAC path end-to-end through
VeloxHashShuffleWriter.

Was this patch authored or co-authored using generative AI tooling?

Co-authored-by: Claude Sonnet 4.6

github-actions · 2026-04-09T02:26:27Z

Run Gluten Clickhouse CI on x86

… in shuffle

github-actions · 2026-04-09T02:27:36Z

Run Gluten Clickhouse CI on x86

FelixYBW · 2026-04-09T18:02:14Z

We have enabled this in Gazelle. @marin-ma do you still remember why we didn't introduce it in Gluten?

github-actions · 2026-04-10T01:26:19Z

Run Gluten Clickhouse CI on x86

marin-ma · 2026-04-10T15:52:05Z

@FelixYBW We used the compression and arrow ipc payload API in gazelle, and added the FastPFor compression for integer column types and it's also used as the default compression method for integers in gazelle https://github.com/fast-pack/FastPFOR

Because adding a new compression algorithm for the Arrow API requires extra patches, we removed it mainly due to maintenance concerns. Using the default LZ4 algorithm did not result in significant performance regression.

guowangy · 2026-04-13T01:13:54Z

@marin-ma We have also investigated FastPFor library, but it only supports int32 type. This version is aimed to optimize int64 data type, which can archive better compression ratio and speed than int32 types.

FelixYBW · 2026-04-15T06:17:27Z

Thank you for your PR. TPCDS perf looks good. SF10T perf boosts by 5%. shuffle size decrease by 7%. most of the queries benefit. compress and decompress time contrinutes most of the perf improvement.

Did you implemented the FFor codec or refered from some other github project?

@marin-ma can you take a look of the PR? Let's merge it once done.

guowangy · 2026-04-15T06:40:17Z

@FelixYBW As in ffor.hpp shows, we take FastLanes as reference (src), with below modification:

Simplify the code using C++ template
Using 4 lanes instead of 8 lanes to support shorter data block
More flexible to handle the tailing data that not fit in a whole block
Add the storage format and make it support generic int64 array compress/uncompress

Resolve conflict in VeloxHashShuffleWriter.cc: keep TAC's bufferTypes_ pass-through to InMemoryPayload alongside main's line-length reformat. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

FelixYBW · 2026-04-15T07:42:10Z

@FelixYBW As in ffor.hpp shows, we take FastLanes as reference (src), with below modification:

Simplify the code using C++ template

Using 4 lanes instead of 8 lanes to support shorter data block

More flexible to handle the tailing data that not fit in a whole block

Add the storage format and make it support generic int64 array compress/uncompress

It's MIT license ...

Option A: Header Preservation (Recommended)

Keep the original MIT license header at the top of the specific files you moved, and add a note below it:

Copyright (c) [Year] [Original Author]

Licensed under the MIT License... (original text)

---

Modifications Copyright (c) [Year] [Your Name/Company], licensed under the Apache License, Version 2.0.

github-actions · 2026-04-15T08:18:38Z

Run Gluten Clickhouse CI on x86

guowangy · 2026-04-15T08:20:51Z

@FelixYBW

It's MIT license ...

Option A: Header Preservation (Recommended)

ffor.hpp license header changed at 01d7388.

marin-ma · 2026-04-15T08:39:03Z

@marin-ma We have also investigated FastPFor library, but it only supports int32 type. This version is aimed to optimize int64 data type, which can archive better compression ratio and speed than int32 types.

Since the compression algos are for different data types, does it mean there's no conflict if we also add FastPFor for int32 compression?

guowangy · 2026-04-15T08:40:36Z

@marin-ma We have also investigated FastPFor library, but it only supports int32 type. This version is aimed to optimize int64 data type, which can archive better compression ratio and speed than int32 types.

Since the compression algos are for different data types, does it mean there's no conflict if we also add FastPFor for int32 compression?

Yes.

marin-ma

LGTM. Thanks!

github-actions · 2026-04-15T09:10:00Z

Run Gluten Clickhouse CI on x86

FelixYBW · 2026-04-15T20:56:29Z

ffor.hpp license header changed at 01d7388.

@weiting-chen @zhouyuan Can you take a look if license modification is OK.

looks not: License Header Check / check (pull_request)
License Header Check / check (pull_request)Failing after 23s

FelixYBW · 2026-04-16T01:40:08Z

Interesting, due to the shuffle datasize is changed, some querys' query plan is changed. Some of the query difference comes from query plan changes.

zhouyuan · 2026-04-16T08:10:38Z

@guowangy would you please also fix the C++ format issue?

github-actions · 2026-04-16T09:53:13Z

Run Gluten Clickhouse CI on x86

guowangy · 2026-04-16T09:54:14Z

@guowangy would you please also fix the C++ format issue?

Done

The TAC codec from apache#11894 only covered INT64. DECIMAL(p>18) is stored as 128-bit HugeInt and previously fell through to LZ4, missing a structural compression opportunity: in OLAP workloads (TPC-H prices, taxes, ...) DECIMAL values usually fit in INT64, so the high 64 bits are 0 and the low 64 bits are narrow -- exactly the pattern FFOR exploits. Implementation: split each 16B value into lo/hi uint64 sub-streams via stride-2 gather into stack-allocated scratch buffers, then run the existing 64-bit FFOR encoder on each. Wire format reuses the 64-bit per-block (bw, count, base) header twice -- the stream is self- describing, no hi/lo length prefix needed. hi sub-streams that are all equal to base degenerate to just the 16B header (bw=0). Velox HUGEINT is mapped to tac::kUInt128; shuffle writer and frame format unchanged. Results vs LZ4 on the same int128 input: compression ratio 0.116 (TAC) vs 0.193 (LZ4) -- 40% smaller End-to-end on TPC-H SF=6000, the wins concentrate on the queries with heavy decimal-keyed shuffles: q15: shuffle size -15%, latency -8% q17: shuffle size -8%, latency -3% q18: shuffle size -8%, latency -3% Generated-by: Claude claude-opus-4-7 Co-authored-by: Guo Wangyang <wangyang.guo@intel.com> Co-authored-by: Lipeng Zhu <lipeng.zhu@intel.com> Signed-off-by: huhengrui <hengrui.hu@intel.com>

* [VL]:shuffle: extend TypeAwareCompress to INT128 (DECIMAL HUGEINT) The TAC codec from #11894 only covered INT64. DECIMAL(p>18) is stored as 128-bit HugeInt and previously fell through to LZ4, missing a structural compression opportunity: in OLAP workloads (TPC-H prices, taxes, ...) DECIMAL values usually fit in INT64, so the high 64 bits are 0 and the low 64 bits are narrow -- exactly the pattern FFOR exploits. Implementation: split each 16B value into lo/hi uint64 sub-streams via stride-2 gather into stack-allocated scratch buffers, then run the existing 64-bit FFOR encoder on each. Wire format reuses the 64-bit per-block (bw, count, base) header twice -- the stream is self- describing, no hi/lo length prefix needed. hi sub-streams that are all equal to base degenerate to just the 16B header (bw=0). Velox HUGEINT is mapped to tac::kUInt128; shuffle writer and frame format unchanged. Results vs LZ4 on the same int128 input: compression ratio 0.116 (TAC) vs 0.193 (LZ4) -- 40% smaller End-to-end on TPC-H SF=6000, the wins concentrate on the queries with heavy decimal-keyed shuffles: q15: shuffle size -15%, latency -8% q17: shuffle size -8%, latency -3% q18: shuffle size -8%, latency -3% Generated-by: Claude claude-opus-4-7 Co-authored-by: Guo Wangyang <wangyang.guo@intel.com> Co-authored-by: Lipeng Zhu <lipeng.zhu@intel.com> Signed-off-by: huhengrui <hengrui.hu@intel.com> * format: fix clang-format violations in PR #12299 clang-format-15 -Werror flagged four spots introduced by 27c53a8 (extend TypeAwareCompress to INT128). No semantic change. - ffor.hpp:354 writeHeader call: argument-pack break style. - ffor.hpp:394 trailing comment missing space after //. - ffor.hpp:473 std::memcpy fits on one line under 120-col limit. - ffor.hpp:667 dst64 assignment fits on one line. To be squashed into 27c53a8 before merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: bound 64-bit FFOR decompress by output buffer capacity decompress64(input, inputSize, output) has no way to know the caller's output buffer capacity, so a corrupt or truncated stream header can drive the decoder to write past the end of `output` before the caller can detect a value-count mismatch. Two specific OOB paths exposed today: - Tail block: `count` is read straight from the wire (`inPtr[1]`). Previously gated only by `inPtr + tailBytes <= inEnd`, so a header with count=0xFF copies 255*8 = 2040 bytes into output regardless of how much room the caller actually has. - Regular block: `blockVals = inPtr[1] * kLanes` is gated only by `<= kMaxValuesPerBlock`. A maximum-sized block is happily decoded even when the caller's buffer is much smaller, overflowing output. Either case is a heap/stack OOB write triggered by malformed shuffle bytes. The 128-bit path already guards against this via outValuesMax (ffor.hpp L634/L642); this brings the 64-bit path to parity. Changes: - decompress64{,Impl}: add `outputSize`, derive `outValuesMax = outputSize / sizeof(uint64_t)`, and reject any tail `count` or block `blockVals` that would exceed `outValuesMax - nDecoded`. - FForCodec::decompress: forward the existing `outputSize` arg (previously dropped at the call site) into ffor::decompress64. - Tests: update the 5 existing decompress64 call sites to pass the output buffer capacity. No behavior change for well-formed streams. On malformed input, the loop stops cleanly at the first oversized header instead of writing past `output`; callers still detect the short result via the existing nDecoded vs expected-value-count check in TypeAwareCompressCodec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Guo Wangyang <wangyang.guo@intel.com> Co-authored-by: Lipeng Zhu <lipeng.zhu@intel.com> Signed-off-by: huhengrui <hengrui.hu@intel.com> --------- Signed-off-by: huhengrui <hengrui.hu@intel.com> Co-authored-by: Guo Wangyang <wangyang.guo@intel.com> Co-authored-by: Lipeng Zhu <lipeng.zhu@intel.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-actions Bot added CORE works for Gluten Core VELOX DOCS labels Apr 9, 2026

guowangy changed the title ~~shuffle: TypeAwareCompress(tac) for column-wise data compression like (U)INT64~~ [VL] shuffle: TypeAwareCompress(tac) for column-wise data compression like (U)INT64 Apr 9, 2026

TypeAwareCompress(tac) for column-wise data compression like (U)INT64…

6d5f57f

… in shuffle

guowangy force-pushed the type-aware-compress branch from d4db9f6 to 6d5f57f Compare April 9, 2026 02:27

cpp code style fix

f462ecf

Merge branch 'main' into type-aware-compress

1337c11

Resolve conflict in VeloxHashShuffleWriter.cc: keep TAC's bufferTypes_ pass-through to InMemoryPayload alongside main's line-length reformat. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

guowangy added 2 commits April 15, 2026 16:01

cpp code style fix

2f6c5e7

Change ffor.cpp license header

01d7388

marin-ma approved these changes Apr 15, 2026

View reviewed changes

Comment thread cpp/velox/shuffle/VeloxHashShuffleWriter.h Outdated

rename: bufferTypes_ -> tacBufferTypes_

3fe769d

FelixYBW approved these changes Apr 15, 2026

View reviewed changes

zhouyuan reviewed Apr 16, 2026

View reviewed changes

Comment thread cpp/core/utils/tac/ffor.hpp

guowangy added 2 commits April 16, 2026 17:37

Move Apache license to header

0943205

cpp code style fix

450ccfa

FelixYBW merged commit f805f13 into apache:main Apr 19, 2026
318 of 337 checks passed

luis4a0 mentioned this pull request Jun 2, 2026

TAC improvements #12220

Draft

hhr293 mentioned this pull request Jun 15, 2026

[VL]:shuffle: extend TypeAwareCompress to INT128 (DECIMAL HUGEINT) #12299

Merged

Uh oh!

Conversation

guowangy commented Apr 9, 2026

What changes are proposed in this pull request?

New files

Shuffle integration

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

FelixYBW commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026

Uh oh!

marin-ma commented Apr 10, 2026

Uh oh!

guowangy commented Apr 13, 2026

Uh oh!

FelixYBW commented Apr 15, 2026

Uh oh!

guowangy commented Apr 15, 2026

Uh oh!

FelixYBW commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 15, 2026

Uh oh!

guowangy commented Apr 15, 2026

Uh oh!

marin-ma commented Apr 15, 2026

Uh oh!

guowangy commented Apr 15, 2026

Uh oh!

marin-ma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Apr 15, 2026

Uh oh!

FelixYBW commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FelixYBW commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zhouyuan commented Apr 16, 2026

Uh oh!

github-actions Bot commented Apr 16, 2026

Uh oh!

guowangy commented Apr 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

FelixYBW commented Apr 15, 2026 •

edited

Loading

FelixYBW commented Apr 15, 2026 •

edited

Loading

FelixYBW commented Apr 16, 2026 •

edited

Loading