Skip to content

[GLUTEN-11599][VL] Support approx_count_distinct_for_intervals function#11729

Closed
malinjawi wants to merge 6 commits into
apache:mainfrom
malinjawi:gluten-11599-approx-count-distinct-for-intervals
Closed

[GLUTEN-11599][VL] Support approx_count_distinct_for_intervals function#11729
malinjawi wants to merge 6 commits into
apache:mainfrom
malinjawi:gluten-11599-approx-count-distinct-for-intervals

Conversation

@malinjawi

Copy link
Copy Markdown
Contributor

Summary

  • Map Spark approx_count_distinct_for_intervals to the Velox backend aggregate

Context

Testing

  • Not run (mapping‑only change)

@github-actions github-actions Bot added CORE works for Gluten Core DOCS labels Mar 9, 2026
@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@jinchengchenghh

Copy link
Copy Markdown
Contributor

Please update here to verify you Velox PR, https://github.com/apache/incubator-gluten/blob/main/ep/build-velox/src/get-velox.sh#L28, and make sure your native implementation is really called. Provide an example test in CI, e.g. warning message disappeared.

[main] WARN org.apache.spark.sql.execution.GlutenFallbackReporter - Validation failed for plan: ObjectHashAggregate[QueryId=113], due to: 
 - Validation failed with exception from: RegularHashAggregateExecTransformer, reason: Could not find a valid substrait mapping name for approx_count_distinct_for_intervals(s_store_sk#87088L, org.apache.spark.sql.catalyst.expressions.UnsafeArrayData@3915186b, 0.05, 0, 0).

@jinchengchenghh jinchengchenghh changed the title [GLUTEN-11599][VL] Map approx_count_distinct_for_intervals [GLUTEN-11599][VL] Support approx_count_distinct_for_intervals function Mar 10, 2026
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@malinjawi

Copy link
Copy Markdown
Contributor Author

Please update here to verify you Velox PR, https://github.com/apache/incubator-gluten/blob/main/ep/build-velox/src/get-velox.sh#L28, and make sure your native implementation is really called. Provide an example test in CI, e.g. warning message disappeared.

[main] WARN org.apache.spark.sql.execution.GlutenFallbackReporter - Validation failed for plan: ObjectHashAggregate[QueryId=113], due to: 
 - Validation failed with exception from: RegularHashAggregateExecTransformer, reason: Could not find a valid substrait mapping name for approx_count_distinct_for_intervals(s_store_sk#87088L, org.apache.spark.sql.catalyst.expressions.UnsafeArrayData@3915186b, 0.05, 0, 0).

Updated locally to address this review. I pinned Gluten’s Velox fetch script to verify against Velox PR 16595 in get-velox.sh, and I added a Spark 3.5 fallback regression that runs ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS and asserts there are 0 fallback nodes. This covers the previous approx_count_distinct_for_intervals validation failure path ("Could not find a valid substrait mapping name ...") and verifies the native implementation is used instead of falling back.

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions Bot added the stale stale label May 29, 2026
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.

@github-actions github-actions Bot closed this Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core DOCS stale stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] Add support for approx_count_distinct_for_intervals

2 participants