Skip to content

Commit b14a5fe

Browse files
authored
feat(cli): add periodic Fuseki TDB2 compaction sidecar to local stack (#1058)
Generated local deployments now include a fuseki-compact service that periodically calls Fuseki's admin compact endpoint (deleteOld=true) to reclaim on-disk space from TDB2's retained historical/deleted data and keep query latency stable. The cadence is configurable via FUSEKI_COMPACT_INTERVAL_SECONDS (default: 86400s / 1 day), added to DEFAULT_ENV_VALUES so it is also backfilled into existing .env files on 'abi deploy local'.
1 parent f0d0f8f commit b14a5fe

3 files changed

Lines changed: 35 additions & 0 deletions

File tree

libs/naas-abi-cli/naas_abi_cli/cli/deploy/local.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@
2929
"PUBLIC_WEB_HOST": "nexus.localhost",
3030
"PUBLIC_API_HOST": "api.localhost",
3131
"RABBITMQ_USER": "abi",
32+
# How often (in seconds) the fuseki-compact sidecar compacts the TDB2
33+
# database to reclaim disk from historical/deleted data. Default: 1 day.
34+
"FUSEKI_COMPACT_INTERVAL_SECONDS": "86400",
3235
"NEXUS_WEB_IMAGE": "ghcr.io/jupyter-naas/nexus-web",
3336
"NEXUS_WEB_TAG": "latest",
3437
"NEXUS_WEB_PORT": "3042",

libs/naas-abi-cli/naas_abi_cli/cli/deploy/templates/local/.env

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ RABBITMQ_USER=abi
5252
RABBITMQ_PASSWORD={{ RABBITMQ_PASSWORD }}
5353

5454
FUSEKI_ADMIN_PASSWORD={{ FUSEKI_ADMIN_PASSWORD }}
55+
# How often (seconds) the fuseki-compact sidecar compacts the TDB2 store. Default: 1 day.
56+
FUSEKI_COMPACT_INTERVAL_SECONDS={{ FUSEKI_COMPACT_INTERVAL_SECONDS }}
5557

5658
PUBLIC_WEB_HOST={{ PUBLIC_WEB_HOST }}
5759
PUBLIC_API_HOST={{ PUBLIC_API_HOST }}

libs/naas-abi-cli/naas_abi_cli/cli/deploy/templates/local/docker-compose.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,36 @@ services:
242242
networks:
243243
- abi-network
244244

245+
# -----------------------------------------------------------------------------
246+
# Fuseki Compaction - Periodic TDB2 database compaction
247+
# -----------------------------------------------------------------------------
248+
# TDB2 retains previous versions of the data after updates/deletes, so the
249+
# on-disk database keeps growing. This lightweight sidecar periodically calls
250+
# Fuseki's admin compact endpoint (deleteOld=true) to reclaim that space and
251+
# keep query latency stable. The cadence is controlled by
252+
# FUSEKI_COMPACT_INTERVAL_SECONDS (default: 86400s = 1 day).
253+
# NOTE: "$$" is how docker-compose escapes a literal "$" so the shell (not
254+
# compose) expands the variable and the "/$/compact" admin path is preserved.
255+
fuseki-compact:
256+
image: curlimages/curl:latest
257+
pull_policy: always
258+
restart: unless-stopped
259+
depends_on:
260+
- fuseki
261+
environment:
262+
- ADMIN_PASSWORD=${FUSEKI_ADMIN_PASSWORD}
263+
- COMPACT_INTERVAL_SECONDS=${FUSEKI_COMPACT_INTERVAL_SECONDS}
264+
entrypoint: /bin/sh
265+
command: >-
266+
-c 'while true; do
267+
sleep "$$COMPACT_INTERVAL_SECONDS";
268+
echo "[fuseki-compact] triggering TDB2 compaction (deleteOld=true)...";
269+
curl -fsS -u "admin:$$ADMIN_PASSWORD" -X POST "http://fuseki:3030/$$/compact/ds?deleteOld=true"
270+
|| echo "[fuseki-compact] compaction request failed";
271+
done'
272+
networks:
273+
- abi-network
274+
245275
# -----------------------------------------------------------------------------
246276
# YasGUI - SPARQL Query Interface
247277
# -----------------------------------------------------------------------------

0 commit comments

Comments
 (0)