Skip to content

Commit 62ef6ff

Browse files
committed
Add an APPEND option to buildParquetCopySQL
buildParquetCopySQL() gains an `append` argument that emits the DuckDB `APPEND` copy option. A `COPY ... PARTITION_BY` into a non-empty target otherwise fails with "Directory ... is not empty"; `APPEND` writes new files into the existing hive-partitioned directory (with unique names, preserving the files earlier parts wrote), which is what a coord-array append needs. Default is FALSE, so the flat/lazy and fresh-write paths are unchanged; DuckDBArray's partitioned writer passes append = TRUE for the append case. Bump version to 0.99.6; update NEWS.
1 parent 6e290f8 commit 62ef6ff

4 files changed

Lines changed: 16 additions & 3 deletions

File tree

DESCRIPTION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
Package: DuckDBDataFrame
2-
Version: 0.99.5
2+
Version: 0.99.6
33
Date: 2026-07-22
44
Title: DuckDB-Backed DataFrame and Table Structures
55
Description:

NEWS.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
# DuckDBDataFrame 0.99.6
2+
3+
## New features
4+
5+
- `buildParquetCopySQL()` gains an `append` argument that emits the DuckDB
6+
`APPEND` copy option. This lets a `PARTITION_BY` write add new files to an
7+
existing (hive-partitioned) directory instead of failing "directory is not
8+
empty", which is what a coord-array append needs (DuckDBArray).
9+
110
# DuckDBDataFrame 0.99.5
211

312
## Bug fixes

R/parquet-io.R

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -348,7 +348,7 @@ quoteSQLColumns <- function(conn, cols) {
348348
#' @rdname parquet-io
349349
buildParquetCopySQL <-
350350
function(query_sql, target_path, order_cols = NULL, partition_by = NULL,
351-
row_group_size = NULL)
351+
row_group_size = NULL, append = FALSE)
352352
{
353353
order_clause <- if (!is.null(order_cols) && length(order_cols) > 0L) {
354354
sprintf(" ORDER BY %s", paste(order_cols, collapse = ", "))
@@ -369,6 +369,9 @@ function(query_sql, target_path, order_cols = NULL, partition_by = NULL,
369369
options <- c(options,
370370
sprintf("PARTITION_BY (%s)", paste(partition_by, collapse = ", ")))
371371
}
372+
if (isTRUE(append)) {
373+
options <- c(options, "APPEND")
374+
}
372375

373376
sprintf(
374377
"COPY (%s%s) TO '%s' (%s)",

man/parquet-io.Rd

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)