Skip to content

Commit 22e9658

Browse files
committed
2 parents 680acc1 + a7696be commit 22e9658

76 files changed

Lines changed: 941 additions & 706 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

_includes/list_of_community_extensions.md

Lines changed: 65 additions & 57 deletions
Large diffs are not rendered by default.

_posts/2025-09-16-announcing-duckdb-140.md

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -240,20 +240,13 @@ A separate blog post will be coming.
240240

241241
## Performance and Optimizations
242242

243-
### Sorting Rework
244-
245-
[Laurens (@lnkuiper)](https://github.com/lnkuiper) [rewrote DuckDB’s sorting implementation](https://github.com/duckdb/duckdb/pull/17584#thread-scaling-performance) ([again](https://github.com/duckdb/duckdb/pull/1561)). This new implementation uses a k-way merge sort to reduce data movement. It is also adaptive to pre-sorted data and uses a new API that makes it possible to use this new sorting code elsewhere in DuckDB, for example in window functions. We are seeing much better thread scaling performance with this implementation. We will publish a separate blog post with more detailed performance measurements.
246-
247-
### Materializing Common Table Expressions
248-
249-
Common table expressions (CTEs) are now materialized by default (instead of inlining them). This both improves performance and resolves some correctness bugs that happened due to inlining.
250-
This feature was [implemented](https://github.com/duckdb/duckdb/pull/17459) by [Denis Hirn (kryonix)](https://github.com/kryonix), who [contributed support for recursive CTEs](https://github.com/duckdb/duckdb/pull/404) back in 2020.
243+
DuckDB v1.4.0 received a number of performance optimizations.
251244

252245
### Checkpointing In-Memory Tables
253246

254247
In-memory tables now support [checkpointing](https://github.com/duckdb/duckdb/pull/18348). This has two key benefits:
255248

256-
* In-memory tables now support compression. This is disabled by default – you can turn it on using:
249+
* In-memory tables now support compression, which can have a significant (5-10×) performance boost for some queries. This is disabled by default – you can turn it on using:
257250

258251
```sql
259252
ATTACH ':memory:' AS memory_compressed (COMPRESS);
@@ -262,6 +255,21 @@ In-memory tables now support [checkpointing](https://github.com/duckdb/duckdb/pu
262255

263256
* Checkpointing triggers vacuuming deleted rows, allowing space to be reclaimed after deletes/truncation.
264257

258+
### Sorting Rework
259+
260+
[Laurens (@lnkuiper)](https://github.com/lnkuiper) [rewrote DuckDB’s sorting implementation](https://github.com/duckdb/duckdb/pull/17584#thread-scaling-performance) ([again](https://github.com/duckdb/duckdb/pull/1561)). This new implementation uses a k-way merge sort to reduce data movement. It is also adaptive to pre-sorted data and uses a new API that makes it possible to use this new sorting code elsewhere in DuckDB, for example in window functions. We are seeing much better thread scaling performance with this implementation. We will publish a separate blog post with more detailed performance measurements.
261+
262+
> Update We have now covered this in a [separate blog post]({% post_url 2025-09-24-sorting-again %}).
263+
264+
### Materializing Common Table Expressions
265+
266+
Common table expressions (CTEs) are now materialized by default (instead of inlining them). This both improves performance and resolves some correctness bugs that happened due to inlining.
267+
This feature was [implemented](https://github.com/duckdb/duckdb/pull/17459) by [Denis Hirn (kryonix)](https://github.com/kryonix), who [contributed support for recursive CTEs](https://github.com/duckdb/duckdb/pull/404) back in 2020.
268+
269+
### Additional Optimizations
270+
271+
DuckBB v1.4.0 fixes some [scaling issues](https://github.com/duckdb/duckdb/pull/17985), supports [caching for hashes of string dictionaries](https://github.com/duckdb/duckdb/pull/18580) and has several [aggregation optimizations](https://github.com/duckdb/duckdb/pull/17718).
272+
265273
## Distribution
266274

267275
### macOS Notarization

_posts/2025-10-07-announcing-duckdb-141.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ excerpt: "Today we are releasing DuckDB 1.4.1, the first bugfix release of our L
88
tags: ["release"]
99
---
1010

11-
In this blog post, we highlight a few important fixes and convenience improvements in DuckDB v1.4.1 LTS.
11+
In this blog post, we highlight a few important fixes and convenience improvements in DuckDB v1.4.1, the first bugfix release in [DuckDB's 1.4 LTS line]({% post_url 2025-09-16-announcing-duckdb-140 %}).
1212
You can find the complete [release notes on GitHub](https://github.com/duckdb/duckdb/releases/tag/v1.4.1).
1313

1414
To install the new version, please visit the [installation page]({% link install/index.html %}).
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
layout: post
3+
title: "Benchmark Results for DuckDB v1.4 LTS"
4+
author: "The DuckDB team"
5+
thumb: "/images/blog/thumbs/benchmark-results-1-4-lts.svg"
6+
image: "/images/blog/thumbs/benchmark-results-1-4-lts.png"
7+
excerpt: "DuckDB v1.4 LTS is both fast and scalable. In in-memory mode, it is the fastest system on ClickBench. In disk-based mode, it can run complex analytical queries on a dataset equivalent to 100 TB CSV files on a single machine."
8+
tags: ["release"]
9+
---
10+
11+
## ClickBench
12+
13+
Today, DuckDB hit #1 on the popular [ClickBench database benchmark](https://benchmark.clickhouse.com/):
14+
15+
<img src="/images/blog/clickbench-top1-light.png"
16+
alt="ClickBench results as of October 9, 2025"
17+
width="800"
18+
class="lightmode-img"
19+
/>
20+
<img src="/images/blog/clickbench-top1-dark.png"
21+
alt="ClickBench results as of October 9, 2025"
22+
width="800"
23+
class="darkmode-img"
24+
/>
25+
26+
This result was made possible due several [performance optimizations]({% post_url 2025-09-16-announcing-duckdb-140 %}#performance-and-optimizations) in DuckDB v1.4.
27+
28+
## TPC-H SF100,000
29+
30+
DuckDB is not only fast but it is also scalable. We have recently run the queries of the [TPC-H workload]({% link docs/stable/core_extensions/tpch.md %}) on the SF100,000 dataset, which is equivalent to 100,000 GB of CSV files.
31+
32+
We ran the experiment on an [`i8g.48xlarge` EC2 instance](https://aws.amazon.com/ec2/instance-types/i8g/), which has 1.5 TB of RAM and 192 CPU cores (AWS Graviton4, Arm64). This instance has 12 NVMe SSD disks, each 3750 GB in size. We created a RAID-0 array from them to have a single 45 TB partition and formatted it using [XFS]({% link docs/stable/guides/performance/environment.md %}#local-disk).
33+
34+
We generated the dataset with the [tpchgen-rs](https://github.com/clflushopt/tpchgen-rs/) tool, a pure Rust implementation of the TPC-H generator. We configured the generator to produce chunks of Parquet files and loaded them into DuckDB. The final DuckDB database was about 27 TB in size (as a single file!).
35+
36+
DuckDB completed all 22 queries of the benchmark using its [larger-than-memory processing]({% post_url 2024-07-09-memory-management %}). For some queries, this required spilling _about 7 terabytes of data_ to disk.
37+
The median query runtime was 1.19 hours and the geometric mean runtime was 1.13 hours.
38+
39+
We will publish a detailed write-up on this experiment in the coming weeks.

_sass/pages/_home.scss

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -701,7 +701,7 @@ body.landing{
701701
right: 0px;
702702
}
703703

704-
section.clickbench {
704+
section.most_scalable {
705705
.flexwrap {
706706
display: flex;
707707
flex-wrap: wrap;
@@ -789,7 +789,7 @@ body.landing{
789789

790790

791791
@media only screen and (max-width: 1100px) {
792-
body.landing section.clickbench {
792+
body.landing section.most_scalable {
793793
.flexwrap {
794794
display: flex;
795795
flex-wrap: wrap;
@@ -921,7 +921,7 @@ body.landing{
921921
max-width: unset;
922922
}
923923
}
924-
body.landing section.clickbench .box > div:first-child{
924+
body.landing section.most_scalable .box > div:first-child{
925925
max-width: 100px;
926926
}
927927
}
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
warning: DO NOT CHANGE THIS MANUALLY, THIS IS GENERATED BY https://github/duckdb/community-extensions repository, check README there
3+
title: a5
4+
excerpt: |
5+
DuckDB Community Extensions
6+
Hierarchical pentagonial indexing for geospatial data that is equal-area and millimeter accurate.
7+
8+
docs:
9+
extended_description: |
10+
For more information regarding usage, see the [documentation](https://query.farm/duckdb_extension_a5.html).
11+
extension:
12+
build: cmake
13+
description: Hierarchical pentagonial indexing for geospatial data that is equal-area and millimeter accurate.
14+
language: C++
15+
license: Apache-2.0
16+
maintainers:
17+
- rustyconover
18+
name: a5
19+
requires_toolchains: rust
20+
excluded_platforms: "wasm_mvp;wasm_eh;wasm_threads;"
21+
version: 2025101102
22+
repo:
23+
github: query-farm/a5
24+
ref: b61ebc04de40e9e36d150c9743f4e80a518ca99a
25+
26+
extension_star_count: 2
27+
extension_star_count_pretty: 2
28+
extension_download_count: null
29+
extension_download_count_pretty: n/a
30+
image: '/images/community_extensions/social_preview/preview_community_extension_a5.png'
31+
layout: community_extension_doc
32+
---
33+
34+
### Installing and Loading
35+
```sql
36+
INSTALL {{ page.extension.name }} FROM community;
37+
LOAD {{ page.extension.name }};
38+
```
39+
40+
{% if page.docs.hello_world %}
41+
### Example
42+
```sql
43+
{{ page.docs.hello_world }}```
44+
{% endif %}
45+
46+
{% if page.docs.extended_description %}
47+
### About {{ page.extension.name }}
48+
{{ page.docs.extended_description }}
49+
{% endif %}
50+
51+
### Added Functions
52+
53+
<div class="extension_functions_table"></div>
54+
55+
| function_name | function_type | description | comment | examples |
56+
|---------------|---------------|-------------|---------|----------|
57+
| a5_area | scalar | NULL | NULL | NULL |
58+
| a5_boundary | scalar | NULL | NULL | NULL |
59+
| a5_cell | scalar | NULL | NULL | NULL |
60+
| a5_children | scalar | NULL | NULL | NULL |
61+
| a5_lon_lat | scalar | NULL | NULL | NULL |
62+
| a5_num_cells | scalar | NULL | NULL | NULL |
63+
| a5_parent | scalar | NULL | NULL | NULL |
64+
| a5_res0_cells | scalar | NULL | NULL | NULL |
65+
| a5_resolution | scalar | NULL | NULL | NULL |
66+
67+

community_extensions/extensions/airport.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,7 @@ excerpt: |
66
The Airport extension brings Arrow Flight support to DuckDB, enabling DuckDB to query, modify, and store data from Arrow Flight servers.
77
88
docs:
9-
extended_description: |
10-
For more information regarding usage, see the [documentation](https://query.farm/duckdb_extension_airport.html).
9+
extended_description: For more information regarding usage, see the [documentation](https://query.farm/duckdb_extension_airport.html).
1110
extension:
1211
build: cmake
1312
description: The Airport extension brings Arrow Flight support to DuckDB, enabling
@@ -21,13 +20,13 @@ extension:
2120
requires_toolchains: parser_tools
2221
test_config: "{\"test_env_variables\":\n {\n \"AIRPORT_TEST_SERVER\": \"grpc+tls://airport-ci.query.farm\"\
2322
\n }\n}\n"
24-
version: 2025091801
23+
version: '2025101201'
2524
repo:
2625
github: query-farm/airport
27-
ref: 0d7b873a0fdf276e9a42e5716fdd872497c4d048
26+
ref: 5872b7de86764abe0daeeeccfd3ad4495f5509c5
2827

29-
extension_star_count: 302
30-
extension_star_count_pretty: 302
28+
extension_star_count: 304
29+
extension_star_count_pretty: 304
3130
extension_download_count: 409
3231
extension_download_count_pretty: 409
3332
image: '/images/community_extensions/social_preview/preview_community_extension_airport.png'

community_extensions/extensions/arrow.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,8 @@ docs:
4343
4444
4545
46-
extension_star_count: 47
47-
extension_star_count_pretty: 47
46+
extension_star_count: 48
47+
extension_star_count_pretty: 48
4848
extension_download_count: 5427
4949
extension_download_count_pretty: 5.4k
5050
image: '/images/community_extensions/social_preview/preview_community_extension_nanoarrow.png'

community_extensions/extensions/bigquery.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ excerpt: |
88
extension:
99
name: bigquery
1010
description: Integrates DuckDB with Google BigQuery, allowing direct querying and management of BigQuery datasets
11-
version: 0.5.0
11+
version: 0.5.1
1212
language: C++
1313
build: cmake
1414
license: MIT
@@ -20,7 +20,7 @@ extension:
2020

2121
repo:
2222
github: hafenkran/duckdb-bigquery
23-
ref: 621dbb517c11d82c0d7693c362591d156ab3da0c
23+
ref: 385d9c46d388e75a7ef620104bf8b543fd351029
2424

2525
docs:
2626
hello_world: |

community_extensions/extensions/bitfilters.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,21 @@ excerpt: |
66
Provides high-performance, space-efficient probabilistic data structures—including quotient, XOR, and binary fuse filters—for fast approximate set membership testing with no false negatives and configurable false positive rates.
77
88
docs:
9-
extended_description: |
10-
For more information regarding usage, see the [documentation](https://query.farm/duckdb_extension_bitfilters.html).
9+
extended_description: For more information regarding usage, see the [documentation](https://query.farm/duckdb_extension_bitfilters.html).
1110
extension:
1211
build: cmake
13-
description: Provides high-performance, space-efficient probabilistic data structures—including quotient, XOR, and binary fuse filters—for fast approximate set membership testing with no false negatives and configurable false positive rates.
12+
description: "Provides high-performance, space-efficient probabilistic data structures\u2014\
13+
including quotient, XOR, and binary fuse filters\u2014for fast approximate set\
14+
\ membership testing with no false negatives and configurable false positive rates."
1415
language: C++
1516
license: MIT
1617
maintainers:
17-
- rustyconover
18+
- rustyconover
1819
name: bitfilters
19-
version: 2025091601
20+
version: '2025101201'
2021
repo:
2122
github: query-farm/bitfilters
22-
ref: cfcf8cf24d7299aa07d0a24c234e5c1f31d04065
23+
ref: 002ab751fc30d94d005b2db27c9f68271b8a36b5
2324

2425
extension_star_count: 2
2526
extension_star_count_pretty: 2

0 commit comments

Comments
 (0)