[VL] Add ANSI mode CI workflow with AI-powered analysis

[baibaichen]

Summary

• Add issue_comment-triggered CI workflow for ANSI SQL mode testing (/analyze-ansi)
• Runs backends-velox + spark-ut tests with SPARK_ANSI_SQL_MODE=true for spark40/spark41
• AI-powered analysis: parses surefire XML, three-color classification (🟢offload 🟡failed 🔴fallback), calls GitHub Models API, posts report as PR comment + Job Summary

Test plan

• Comment /analyze-ansi on this PR to trigger workflow
• Verify build completes (native lib + test compilation)
• Verify test jobs run with ANSI mode enabled (failures expected)
• Verify analysis report posted as PR comment with three-color breakdown
• Verify Job Summary contains same report
• Confirm get-velox.sh unchanged from main

🤖 Generated with Claude Code

[baibaichen]

/analyze-ansi

[github-actions]

🔄 ANSI mode analysis started by @baibaichen. View run

[github-actions]

ANSI Mode Test Analysis Report

Overview

Total tests: 33746

Classification	Count
🟢 Passed+Offload	27231
🟡 Failed+Offload	78
⚪ Skipped	6437

Per-Job Breakdown

spark-test-backends-velox-ansi-report

• Failed+Offload: 6
• Passed+Offload: 748
• Skipped: 21

spark-test-backends-velox-ansi-spark40-report

• Failed+Offload: 6
• Passed+Offload: 751
• Skipped: 21

spark-test-spark-ut-ansi-report

• Failed+Offload: 33
• Passed+Offload: 12636
• Skipped: 4169

spark-test-spark-ut-ansi-spark40-report

• Failed+Offload: 33
• Passed+Offload: 13096
• Skipped: 2226

Failed Tests (sample, max 50)

Suite	Test	Color	Message
UDFPartialProjectSuite	udf in agg simple	🟡 Failed+Offload	[ARITHMETIC_OVERFLOW] integer overflow. Use 'try_add' to tolerate overflow and return NULL instead.
MiscOperatorSuite	Support multi-children count with row construct	🟡 Failed+Offload	[DIVIDE_BY_ZERO] Division by zero. Use try_divide to tolerate divisor being 0 and return NULL inst
MiscOperatorSuite	Remainder with non-foldable right side	🟡 Failed+Offload	[DIVIDE_BY_ZERO] Division by zero. Use try_divide to tolerate divisor being 0 and return NULL inst
MiscOperatorSuite	Cast string to date	🟡 Failed+Offload	[CAST_INVALID_INPUT] The value '-1' of the type "STRING" cannot be cast to "DATE" because it is malf
FallbackSuite	fallback when nested loop join has unsupported expression	🟡 Failed+Offload	[CAST_INVALID_INPUT] The value '' of the type "STRING" cannot be cast to "BIGINT" because it is malf
DateFunctionsValidateSuite	make_date	🟡 Failed+Offload	[DATETIME_FIELD_OUT_OF_BOUNDS] Invalid value for DayOfMonth (valid values 1 - 28/31): 50. If necessa
GlutenDataFrameAggregateSuite	SPARK-28067: Aggregate sum should not return wrong results for decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenDataFrameAggregateSuite	SPARK-35955: Aggregate avg should not return wrong results for decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenDataFrameAggregateSuite	SPARK-28224: Aggregate sum big decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenFileSourceSQLInsertTestSuite	SPARK-38228: legacy store assignment should not fail on error under ANSI mode	🟡 Failed+Offload	Job aborted due to stage failure: Task 0 in stage 114.0 failed 1 times, most recent failure: Lost ta
GlutenTPCDSV1_4_PlanStabilitySuite	check simplified (tpcds-v1.4/q83)	🟡 Failed+Offload	Plans did not match: last approved simplified plan: /__w/gluten/gluten/gluten-ut/spark40/src/test/r
GlutenSQLQuerySuite	SPARK-39166: Query context of binary arithmetic should be serialized to executors when WSCG is off	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but org.apache.spark.Spar
GlutenSQLQuerySuite	SPARK-39175: Query context of Cast should be serialized to executors when WSCG is off	🟡 Failed+Offload	ex.isInstanceOf[org.apache.spark.SparkNumberFormatException] was false, and org.apache.spark.SparkEx
GlutenSQLQuerySuite	SPARK-39190,SPARK-39208,SPARK-39210: Query context of decimal overflow error should be serialized to executors when WSCG is off	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but no exception was thro
GlutenSQLQuerySuite	SPARK-40389: Don't eliminate a cast which can cause overflow	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but no exception was thro
GlutenCastWithAnsiOffSuite	SPARK-16729 type checking for casting to date type	🟡 Failed+Offload	DataTypeMismatch("CAST_WITH_FUNC_SUGGESTION", Map("srcType" -> ""DOUBLE"", "targetType" -> ""DATE"",
GlutenCastWithAnsiOffSuite	null cast #2	🟡 Failed+Offload	expr.resolved was false
GlutenCastWithAnsiOffSuite	cast string to date #2	🟡 Failed+Offload	Exception evaluating cast(2015-03-18X as date)
GlutenCastWithAnsiOffSuite	casting to fixed-precision decimals	🟡 Failed+Offload	Exception evaluating cast(1003 as decimal(1,-2))
GlutenCastWithAnsiOffSuite	cast string to boolean II	🟡 Failed+Offload	Exception evaluating cast(abc as boolean)
GlutenCastWithAnsiOffSuite	cast from array II	🟡 Failed+Offload	true did not equal false
GlutenCastWithAnsiOffSuite	cast from map II	🟡 Failed+Offload	true did not equal false
GlutenCastWithAnsiOffSuite	cast from struct II	🟡 Failed+Offload	true did not equal false
GlutenCastWithAnsiOffSuite	complex casting	🟡 Failed+Offload	true did not equal false
GlutenCastWithAnsiOffSuite	cast from date	🟡 Failed+Offload	expr.resolved was false
GlutenCastWithAnsiOffSuite	cast a timestamp before the epoch 1970-01-01 00:00:00Z	🟡 Failed+Offload	Exception evaluating cast(1900-05-05 18:34:56.1 as tinyint)
GlutenCastWithAnsiOffSuite	data type casting II	🟡 Failed+Offload	Exception evaluating cast(cast(cast(cast(cast(cast(5 as timestamp) as tinyint) as decimal(38,18)) as
GlutenCastWithAnsiOffSuite	SPARK-35720: cast invalid string input to timestamp without time zone	🟡 Failed+Offload	Exception evaluating cast(00:00:00 as timestamp_ntz)
GlutenCastWithAnsiOffSuite	SPARK-42176: cast boolean to timestamp	🟡 Failed+Offload	expr.resolved was false
GlutenCastWithAnsiOffSuite	Gluten - data type casting	🟡 Failed+Offload	Exception evaluating cast(cast(2015-03-07 23:30:00.342 as string) as timestamp)
GlutenCastWithAnsiOffSuite	cast from boolean to timestamp	🟡 Failed+Offload	expr.resolved was false
GlutenCastWithAnsiOffSuite	Gluten - cast string to timestamp	🟡 Failed+Offload	Exception thrown in awaitResult:
GlutenQueryContextSuite	SPARK-50290: Add a flag to disable DataFrame context	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but org.apache.spark.Spar
GlutenTryCastSuite	Gluten - data type casting	🟡 Failed+Offload	Exception evaluating try_cast(try_cast(2015-03-07 23:30:00.31 as string) as timestamp)
GlutenTryCastSuite	Gluten - cast string to timestamp	🟡 Failed+Offload	Exception thrown in awaitResult:
GlutenQueryExecutionAnsiErrorsSuite	SPARK-46922: user-facing runtime errors	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but org.apache.spark.Spar
GlutenInsertSuite	Gluten - remove v1writes sort	🟡 Failed+Offload	Job aborted due to stage failure: Task 0 in stage 488.0 failed 1 times, most recent failure: Lost ta
VeloxAdaptiveQueryExecSuite	Gluten - SPARK-33551: Do not use AQE shuffle read for repartition	🟡 Failed+Offload	true did not equal false
GlutenTPCDSV1_4_PlanStabilityWithStatsSuite	check simplified sf100 (tpcds-v1.4/q83)	🟡 Failed+Offload	Plans did not match: last approved simplified plan: /__w/gluten/gluten/gluten-ut/spark40/src/test/r
GlutenDataFrameAggregateSuite	SPARK-28067: Aggregate sum should not return wrong results for decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenDataFrameAggregateSuite	SPARK-35955: Aggregate avg should not return wrong results for decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenDataFrameAggregateSuite	SPARK-28224: Aggregate sum big decimal overflow	🟡 Failed+Offload	Expected exception java.lang.ArithmeticException to be thrown, but no exception was thrown
GlutenFileSourceSQLInsertTestSuite	SPARK-38228: legacy store assignment should not fail on error under ANSI mode	🟡 Failed+Offload	Job aborted due to stage failure: Task 0 in stage 114.0 failed 1 times, most recent failure: Lost ta
GlutenTPCDSV1_4_PlanStabilitySuite	check simplified (tpcds-v1.4/q83)	🟡 Failed+Offload	Plans did not match: last approved simplified plan: /__w/gluten/gluten/gluten-ut/spark41/src/test/r
GlutenSQLQuerySuite	SPARK-39166: Query context of binary arithmetic should be serialized to executors when WSCG is off	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but org.apache.spark.Spar
GlutenSQLQuerySuite	SPARK-39175: Query context of Cast should be serialized to executors when WSCG is off	🟡 Failed+Offload	ex.isInstanceOf[org.apache.spark.SparkNumberFormatException] was false, and org.apache.spark.SparkEx
GlutenSQLQuerySuite	SPARK-39190,SPARK-39208,SPARK-39210: Query context of decimal overflow error should be serialized to executors when WSCG is off	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but no exception was thro
GlutenSQLQuerySuite	SPARK-40389: Don't eliminate a cast which can cause overflow	🟡 Failed+Offload	Expected exception org.apache.spark.SparkArithmeticException to be thrown, but no exception was thro
GlutenCastWithAnsiOffSuite	SPARK-16729 type checking for casting to date type	🟡 Failed+Offload	DataTypeMismatch("CAST_WITH_FUNC_SUGGESTION", Map("srcType" -> ""DOUBLE"", "targetType" -> ""DATE"",
GlutenCastWithAnsiOffSuite	null cast #2	🟡 Failed+Offload	expr.resolved was false

...and 28 more failures

AI Analysis

Analysis of Apache Gluten ANSI SQL Mode Test Results Using Velox Backend

1. Key Failure Patterns and Root Causes

• Error propagation under ANSI mode: Several test failures (e.g., arithmetic overflow, division by zero, cast failures) are due to strict adherence to ANSI SQL error handling. Instead of tolerating invalid cases (like overflow or invalid casts) and returning NULL, Velox triggers explicit exceptions such as [ARITHMETIC_OVERFLOW], [DIVIDE_BY_ZERO], [CAST_INVALID_INPUT], or mismatched error types (e.g., SparkNumberFormatException vs. SparkException).
• Precision issues in aggregation operations: Tests for aggregate functions like SUM and AVG reveal that decimal overflow exceptions are not being correctly handled (e.g., SPARK-28067 and SPARK-35955). This likely relates to inadequate handling of overflow detection in high-precision arithmetic for decimal types.
• Type casting issues: Numerous failures stem from CAST operations (e.g., string-to-date, string-to-boolean, or casting arrays, maps, and structs). Problems include exceptions during invalid casts or the lack of proper resolution of expressions (expr.resolved was false). These issues point to incomplete or incorrect implementations of CAST semantics in Velox under ANSI mode.
• Plan stability and executor behavior: A few tests failed due to mismatches in plan output (e.g., tpcds-v1.4/q83) or differences in how exceptions are serialized and propagated to executors under specific configurations (SPARK-39166, SPARK-39175).

2. Expression Types Needing ANSI Implementation Work

• Arithmetic Expressions: Incomplete handling of exceptions such as overflow (e.g., try_add, try_divide) and division by zero.
• Cast Expressions: Several edge cases remain unimplemented or have incorrect behavior, including:
  • CAST to/from complex types (arrays, maps, and structs)
  • String-to-date, string-to-boolean, and string-to-numeric conversions
  • Boundary cases for date, timestamp, and numeric types (e.g., invalid epoch timestamps or extreme date values).
• Aggregation Functions: Overflow detection in aggregate functions like SUM and AVG (especially for high-precision decimals).
• Error Propagation: ANSI-specific exception types (e.g., SparkArithmeticException, ArithmeticException) and error state tracking must properly align with Spark’s ANSI mode semantics.

3. Priority Recommendations for Fixing

1. Address Arithmetic and Decimal Overflow Handling:
   • Implement support for try_add and try_divide for graceful NULL results when errors like overflow or division by zero occur.
   • Ensure aggregate functions properly detect and throw overflow exceptions while adhering to SQLSTATE codes (e.g., 22003 for overflow, 22012 for division by zero).
2. Resolve CAST Compatibility Issues:
   • Focus on resolving critical CAST failures, particularly string-to-date, string-to-boolean, and complex type casting issues. Align behavior with Spark's try_cast semantics when in ANSI mode.
   • Fix scenario-specific issues, like CAST mismatches due to invalid inputs or unresolved expressions (expr.resolved was false).
3. Plan Serialization and Stability:
   • Recheck test plans for consistency. Ensure that serialized query contexts propagate behavior (e.g., correct exceptions) consistently from driver to executors.
4. Enhance Error Propagation Consistency:
   • Ensure correct mapping of Velox exceptions to corresponding Spark-specific ANSI error types and SQLSTATE codes for consistent behavior.

Conclusion

A substantial proportion of failing tests stem from incomplete or inconsistent ANSI SQL mode implementation in Velox, primarily around robust error handling, edge case management in CAST expressions, and arithmetic/aggregation overflow detection. Prioritizing fixes for these areas will significantly improve conformance with ANSI SQL and Spark's semantics.

---

Triggered by @baibaichen | Workflow run