Enhancing Extensibility for subOp-style Plugin Optimizations

[LeiRui]

Enhancing Extensibility for subOp-style Plugin Optimizations

Hi Bolt community!

We're working on ARM SVE optimizations for Hash Aggregation and would like to discuss with the community about enhancing Bolt's extensibility to better support plugin-based optimizations. We've implemented a Hash Aggregation subOp (part of our broader "subOp" plugin framework) that extends Velox-style hash aggregation with SVE vectorization. We're exploring opportunities to work more closely with the Bolt community, and we hope both sides could benefit from enhanced extensibility that enables plugin-based optimizations.

1 Background

1.1 What We've Built: Hash Aggregation subOp

• Implements SVE vectorization for for hash table operations and aggregate functions
• Works as a standalone plugin (.so library) and uses runtime operator replacement via the adapter pattern: we register a DriverAdapter that intercepts query plan execution and replaces HashAggregation operators with our EnhancedHashAggregation implementation.

1.2 The Challenge We Face

To implement our SVE optimizations, we need to:

1. Replace core algorithms with SVE-optimized versions (group probing, aggregate updates)
2. Reuse substantial portions of surrounding infrastructure. For example:
   • HashAggregation's operator lifecycle management (initialization, input processing, output generation)
   • GroupingSet's grouping logic (group management, spill handling)
   • HashTable's infrastructure (e.g., prepareForGroupProbe for deciding hash table mode and computing hashes, probe methods like arrayGroupProbe that haven't been optimized with SVE yet)

Our optimization granularity creates a challenge:

• Not as small as local scalar loop replacements (like [Issue #126]([Feature] Add SVE (Scalable Vector Extension) support for StringCore.h #126), which directly modifies a single function with ~17 lines)
• Not as large as complete operator replacements (like [Velox's Cudf adapter](https://velox-lib.io/blog/extending-velox-with-cudf/), which replaces entire operators independently and doesn't need to reuse core infrastructure)

1.3 Why Plugin Architecture?

We chose a plugin architecture (similar to Velox's Cudf adapter pattern) because it:

• ✅ Isolates experimental code from stable core
• ✅ Enables rapid iteration without affecting core stability
• ✅ Encourages innovation by allowing third-party optimizations (e.g., ARM, GPU, custom algorithms)
• ✅ Supports ecosystem growth by enabling diverse optimization strategies

Why not directly modify core code? Because our optimizations involve:

• Algorithm-level changes: New group probing algorithms
• Function restructuring: Adapting aggregate update logic to fit SIMD paradigms
• Multi-layer changes: Affecting HashAggregation → GroupingSet → HashTable → Aggregate call chain

Direct inline modifications would require substantial changes across multiple layers, mix experimental optimizations with stable core code, and require maintainers to review architecture-specific code they may not use.

2 Current Challenge: Code Duplication

However, unlike Cudf (which replaces entire operators independently), our subOp needs to partially reuse core infrastructure. Currently, because many critical members are private, we've had to duplicate ~5000+ lines of code to access the infrastructure we need to reuse. This creates high maintenance burden, duplicates bug fixes, and makes upgrades difficult. Specifically, we replaces the entire call chain: EnhancedHashAggregation → EnhancedGroupingSet → EnhancedHashTable → EnhancedAggregate, where we replace core algorithms (group probing, aggregate updates) with SVE-optimized versions while reusing surrounding infrastructure.

Example: Our SVE-optimized group probing algorithm may require different hash table layout/sizing, so we need to customize allocateTables(). If we inherit HashTable and override setHashMode (virtual), we can call our customized checkSize (private) → allocateTables (private) → finally needs to modify capacity_, numBuckets_, etc (all private). We can't modify these private members of HashTable, so we'd maintain our own copies, leading to two sets of state that get out of sync. This is why we currently duplicate code instead of inherit HashTable.

3 Our Thinking and Questions

We're exploring whether Bolt could make small changes to its core code to enable third-party plugins to reuse Bolt code through inheritance, rather than duplicating it.

Preliminary approaches we're considering:

• Change some private members to protected (e.g., HashAggregation::groupingSet_, GroupingSet::table_, HashTable::capacity_, HashTable::numBuckets_)
• Change some private methods to protected virtual (e.g., HashTable::checkSize(), HashTable::allocateTables()) to allow derived classes to customize behavior when needed
• Add protected virtual factory methods to allow derived classes to customize component creation (e.g., HashAggregation::createGroupingSet(), GroupingSet::createHashTable())

We're not proposing a fully-baked solution yet - we'd like to understand the community's perspective and constraints first, then work together to find the best approach that fits Bolt's design philosophy. For example, are there concerns about changing private to protected (encapsulation, API stability)? Are there existing patterns in Bolt for similar extension needs?

We're committed to actively participating and contributing code. We'd love to hear the community's thoughts!

[frankobe]

@LeiRui Thanks for the proposal. .so files creates the flexibility for users to reuse bolt capability without leaking their source code, which can be useful for dynamic connectors or user-defined functions.

However, operator is the core part of Bolt compute engine itself so blackbox-like .so files makes it challenging to maintain the source code. The subOp interface is hard to generalize as it can vary per operators.

Are you open to create a pull request containing the source code of SVE-optimized version of HashTable? If yes, we can add a config to argument of GroupingSet::createHashTable() to allow the custimozed implementation of HashTable.

cc @kexianda @guhaiyan0221 @yangzhg

[LeiRui]

Thank you for the feedback and the suggestion to contribute via PR. We really appreciate you taking the time to consider this.

On the config approach: We're open to contributing the SVE HashTable as source code and working with the config in createHashTable() to select the implementation. For clarity: even when subOp is used as a .so plugin, we plan to open source its code in the future—the .so format is for optional loading, not for hiding implementation.

On code reuse: Our original discussion was mainly about extensibility and reuse. HashTable appears to have been designed as a leaf implementation rather than an extensible base class. We only need to customize a few key paths (e.g., groupProbe, insertForGroupBy, allocateTables) for SVE, but to do that we have to duplicate a large amount of surrounding code from HashTable that we could otherwise reuse. We're asking whether Bolt would be open to making HashTable extensible—so that SVEHashTable could inherit from HashTable and override only the hot paths, rather than reimplementing everything from BaseHashTable. This would require small changes (e.g., some private → protected). This is separate from the config approach—the config decides how we plug in; the reuse question is about how we implement without duplication. A simple diagram to illustrate:

We'd be glad to hear your thoughts and are happy to adjust our approach to fit Bolt's design.

[frankobe]

@LeiRui It would be easier to discuss if a PR is available so we know what exactly extra optimization SVE can do. In the PR, you can propose the private-> protected change so let's see whether a new layer of interface like HashTable Extensible(need a better name) needs to be added. The plan will be like follows:

BaseHashTable -> HashTable Extension -> HashTable leaf (existing implementation )
                                            |
                                            ---------------> SVEHashTable

SVEHashTable should be contributed as source code, not .so files.