Commit 4825905
[SPARK-57260][SQL] Fix variable resolution in REPLACE WHERE clause of INSERT INTO
### What changes were proposed in this pull request?
This PR fixes variable resolution in the REPLACE WHERE clause of INSERT INTO statements.
REPLACE WHERE is represented as OverwriteByExpression.deleteExpr during analysis. Previously, this expression was resolved only against the target table output because resolveExpressionByPlanOutput was called without includeLastResort = true.
This PR enables last-resort resolution for OverwriteByExpression.deleteExpr, allowing SQL variables declared with DECLARE to be resolved in REPLACE WHERE predicates while preserving table-column precedence.
### Why are the changes needed?
[SPARK-57260](https://issues.apache.org/jira/browse/SPARK-57260) reports that SQL variables can be used in the VALUES clause of INSERT INTO, but not in the REPLACE WHERE clause.
For example, this previously failed during analysis:
```
BEGIN
DECLARE x INT DEFAULT 1;
INSERT INTO table_y
REPLACE WHERE y = x
VALUES (x);
END
```
The predicate y = x could not resolve x as a SQL variable, resulting in an unresolved column/variable error.
### Does this PR introduce any user-facing change?
Yes.
Before this change, INSERT INTO ... REPLACE WHERE could not resolve SQL variables declared with DECLARE in the REPLACE WHERE predicate and failed during analysis with an unresolved column/variable error.
After this change, INSERT INTO ... REPLACE WHERE can resolve SQL variables declared with DECLARE in the REPLACE WHERE predicate.
### How was this patch tested?
Added test coverage for variable resolution in INSERT INTO ... REPLACE WHERE, including:
session variables
SQL scripting local variables
table-column precedence over SQL scripting variables
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor GPT-5.5 and Claude Code Opus 4.8
Closes #56321 from joelrobin18/SPARK-57260-fix-replace-where-variable-resolution.
Authored-by: Joel Robin P <joelrobin1818@gmail.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
(cherry picked from commit a32cda3)
Signed-off-by: Max Gekk <max.gekk@gmail.com>1 parent 5bee0db commit 4825905
3 files changed
Lines changed: 73 additions & 1 deletion
File tree
- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis
- core/src/test/scala/org/apache/spark/sql
- connector
- scripting
Lines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1698 | 1698 | | |
1699 | 1699 | | |
1700 | 1700 | | |
1701 | | - | |
| 1701 | + | |
| 1702 | + | |
1702 | 1703 | | |
1703 | 1704 | | |
1704 | 1705 | | |
| |||
Lines changed: 23 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3923 | 3923 | | |
3924 | 3924 | | |
3925 | 3925 | | |
| 3926 | + | |
| 3927 | + | |
| 3928 | + | |
| 3929 | + | |
| 3930 | + | |
| 3931 | + | |
| 3932 | + | |
| 3933 | + | |
| 3934 | + | |
| 3935 | + | |
| 3936 | + | |
| 3937 | + | |
| 3938 | + | |
| 3939 | + | |
| 3940 | + | |
| 3941 | + | |
| 3942 | + | |
| 3943 | + | |
| 3944 | + | |
| 3945 | + | |
| 3946 | + | |
| 3947 | + | |
| 3948 | + | |
3926 | 3949 | | |
3927 | 3950 | | |
3928 | 3951 | | |
| |||
Lines changed: 48 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
348 | 348 | | |
349 | 349 | | |
350 | 350 | | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
351 | 399 | | |
352 | 400 | | |
353 | 401 | | |
| |||
0 commit comments