Skip to content

Commit 8d91fb0

Browse files
authored
Update 53 upgrade guide to note release, other changes (#21449)
## Which issue does this PR close? - Related to #19692 ## Rationale for this change @rluvaton noted some issues with the 53 upgrade guide #19692 (comment): > FYI, the migration guide says 53.0.0 was not released yet and it miss the following breaking changes: > > #20319 - removed deprecated function from ExecutionPlan trait > #19635 - added argument to HashJoinExec::try_new function ## What changes are included in this PR? Fix the above ## Are these changes tested? Yes by CI ## Are there any user-facing changes? Better docs
1 parent 249c23c commit 8d91fb0

File tree

1 file changed

+42
-7
lines changed
  • docs/source/library-user-guide/upgrading

1 file changed

+42
-7
lines changed

docs/source/library-user-guide/upgrading/53.0.0.md

Lines changed: 42 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,6 @@
2121

2222
## DataFusion 53.0.0
2323

24-
**Note:** DataFusion `53.0.0` has not been released yet. The information provided
25-
*in this section pertains to features and changes that have already been merged
26-
*to the main branch and are awaiting release in this version. See [#19692] for
27-
\*more details.
28-
29-
[#19692]: https://github.com/apache/datafusion/issues/19692
30-
3124
### Upgrade arrow/parquet to 58.0.0 and object_store to 0.13.0
3225

3326
DataFusion 53.0.0 uses `arrow` and `parquet` 58.0.0, and `object_store` 0.13.0.
@@ -39,6 +32,39 @@ See the [Arrow 58.0.0 release notes] and the [object_store 0.13.0 upgrade guide]
3932
[arrow 58.0.0 release notes]: https://github.com/apache/arrow-rs/releases/tag/58.0.0
4033
[object_store 0.13.0 upgrade guide]: https://github.com/apache/arrow-rs-object-store/blob/v0.13.0/CHANGELOG.md
4134

35+
### `ExecutionPlan::statistics` removed
36+
37+
The deprecated `ExecutionPlan::statistics()` method has been removed. If you
38+
implement custom `ExecutionPlan`s, remove that method from your impl and
39+
implement `partition_statistics()` instead.
40+
41+
**Before:**
42+
43+
```rust,ignore
44+
impl ExecutionPlan for MyExec {
45+
// ...
46+
47+
fn statistics(&self) -> Result<Statistics> {
48+
Ok(Statistics::new_unknown(&self.schema()))
49+
}
50+
}
51+
```
52+
53+
**After:**
54+
55+
```rust,ignore
56+
impl ExecutionPlan for MyExec {
57+
// ...
58+
59+
fn partition_statistics(&self, _partition: Option<usize>) -> Result<Statistics> {
60+
Ok(Statistics::new_unknown(&self.schema()))
61+
}
62+
}
63+
```
64+
65+
If you do not have partition-specific statistics, return the same value for
66+
`None` and for any partition index.
67+
4268
### `ExecutionPlan::properties` now returns `&Arc<PlanProperties>`
4369

4470
Now `ExecutionPlan::properties()` returns `&Arc<PlanProperties>` instead of a
@@ -125,6 +151,15 @@ let sub_plan = self.query_to_plan(subquery, planner_context)?;
125151
planner_context.pop_outer_query_schema();
126152
```
127153

154+
### `HashJoinExec::try_new` adds `null_aware`
155+
156+
`HashJoinExec::try_new` now takes an extra `null_aware: bool` argument. This
157+
flag is used for null-aware anti joins, such as plans generated for `NOT IN`
158+
subqueries.
159+
160+
Most callers should pass `false`, the previous behavior. Pass `true` only for null-aware
161+
`JoinType::LeftAnti` joins.
162+
128163
### `FileSinkConfig` adds `file_output_mode`
129164

130165
`FileSinkConfig` now includes a `file_output_mode: FileOutputMode` field to control

0 commit comments

Comments
 (0)