Fixed bugs:
- fix: cast timestamp to decimal is unsupported #1281 (wForget)
- fix: partially fix consistency issue of hash functions with decimal input #1295 (wForget)
- fix: Improve testing for array_remove and fallback to Spark for unsupported types #1308 (andygrove)
- fix: address post merge comet-parquet-exec review comments #1327 (parthchandra)
- fix: memory pool error type #1346 (kazuyukitanimura)
- fix: Fall back to Spark when hashing decimals with precision > 18 #1325 (andygrove)
- fix: expressions doc for ArrayRemove #1356 (kazuyukitanimura)
- fix: pass scale to DF round in spark_round #1341 (cht42)
- fix: Mark cast from float/double to decimal as incompatible #1372 (andygrove)
- fix: Passthrough condition in StaticInvoke case block #1392 (EmilyMatt)
- fix: disable checking for uint_8 and uint_16 if complex type readers are enabled #1376 (parthchandra)
Performance related:
- perf: improve performance of update metrics #1329 (wForget)
- perf: Use DataFusion FilterExec for experimental native scans #1395 (mbutrovich)
Implemented enhancements:
- feat: Add HasRowIdMapping interface #1288 (viirya)
- feat: Upgrade to DataFusion 45 #1364 (andygrove)
- feat: Add fair unified memory pool #1369 (kazuyukitanimura)
- feat: Add unbounded memory pool #1386 (kazuyukitanimura)
- feat: make random seed configurable in fuzz-testing #1401 (wForget)
- feat: override executor overhead memory only when comet unified memory manager is disabled #1379 (wForget)
Documentation updates:
- docs: Fix links and provide complete benchmarking scripts #1284 (andygrove)
- doc: update memory tuning guide #1394 (kazuyukitanimura)
Other:
- chore: Start 0.6.0 development #1286 (andygrove)
- minor: update compatibility #1303 (kazuyukitanimura)
- chore: extract conversion_funcs, conditional_funcs, bitwise_funcs and array_funcs expressions to folders based on spark grouping #1223 (rluvaton)
- chore: extract math_funcs expressions to folders based on spark grouping #1219 (rluvaton)
- chore: merge comet-parquet-exec branch into main #1318 (andygrove)
- Feat: Support array_intersect function #1271 (erenavsarogullari)
- build(deps): bump pprof from 0.13.0 to 0.14.0 in /native #1319 (dependabot[bot])
- chore: Fix merge conflicts from merging comet-parquet-exec into main #1320 (andygrove)
- chore: Revert accidental re-introduction of off-heap memory requirement #1326 (andygrove)
- chore: Fix merge conflicts from merging comet-parquet-exec into main #1323 (mbutrovich)
- Feat: Support array_join function #1290 (erenavsarogullari)
- Fix missing slash in spark script #1334 (xleoken)
- chore: Refactor QueryPlanSerde to allow logic to be moved to individual classes per expression #1331 (andygrove)
- build: re-enable upload-test-reports for macos-13 runner #1335 (viirya)
- chore: Upgrade to Arrow 53.4.0 #1338 (andygrove)
- Feat: Support arrays_overlap function #1312 (erenavsarogullari)
- chore: Move all array_* serde to new framework, use correct INCOMPAT config #1349 (andygrove)
- chore: Prepare for DataFusion 45 (bump to DataFusion rev 5592834 + Arrow 54.0.0) #1332 (andygrove)
- minor: commit compatibility doc #1358 (kazuyukitanimura)
- minor: update fuzz dependency #1357 (kazuyukitanimura)
- chore: Remove redundant processing from exprToProtoInternal #1351 (andygrove)
- chore: Adding an optional
hdfscrate #1377 (comphead) - chore: Refactor aggregate expression serde #1380 (andygrove)