feat(es): improve Search CLI with new commands and safer index management

Replace elasticsearch-dsl with direct client calls, add `rebuild`, `health`,
and `queue` commands, and fix several correctness issues across the CLI.

* drop `elasticsearch_dsl.Index` from `index.py` and `snapshot/cli.py`; use
  `current_search_client.indices.*` with `allow_no_indices=True` and
  `ignore_unavailable=True` instead
* change default index pattern for `open`/`close` commands from `_all` to `*`
* add `rero es index rebuild <resource>`: zero-data-loss index migration —
  creates a fresh dated index, reindexes, shows old/new doc counts, asks for
  confirmation before switching all aliases and deleting the old index
* add `rero es health`: connectivity check for Search, database, and Celery
  broker; exits 1 if any service is unreachable or in error state
* add `rero es queue list/stats/purge`: inspect Celery workers and queue
  depths; purge a named queue with `--yes-i-know` guard
* fix `raise click.Abort` → `raise click.Abort()` in `slm/cli.py` (bare
  reference did nothing)
* fix dead assertions in `tests/test_tasks.py` (missing `assert` keyword) and
  correct expected output string "name arg" → "named arg"
* fix `__all__ = "rero"` → `__all__ = ["rero"]` in `cli/__init__.py` and
  `cli/utils.py` (string iterates as characters)
* render `rero es snapshot list` as a formatted table (name, start time,
  duration, shards, colour-coded state) instead of raw JSON
* add `--names-only` flag to `snapshot list` for scripting use
* improve `create_snapshot --wait` output: show state, duration and shard
  counts instead of the raw JSON blob
* add `restore_snapshot --delete` flag: deletes RERO ILS indices before
  restoring
* rename "Elasticsearch"/"OpenSearch" → "Search" throughout docstrings and
  comments

Co-Authored-by: Peter Weber <peter.weber@rero.ch>

Author: rerowep

CHANGELOG.md

@@ -35,7 +35,7 @@ SPDX-License-Identifier: AGPL-3.0-or-later
 * feat(dev): add uv and ruff [#21](https://github.com/rero/rero-invenio-base/pull/21) (by @PascalRepond)
 * chore(actions): pypi publish package use poetry [#20](https://github.com/rero/rero-invenio-base/pull/20) (by @PascalRepond)
 
-## [v0.3.1](https://github.com/rero/rero-invenio-base/tree/v0.3.0) (2025-04-16)
+## [v0.3.1](https://github.com/rero/rero-invenio-base/tree/v0.3.1) (2025-04-16)
 
 [Full Changelog](https://github.com/rero/rero-invenio-base/compare/v0.3.0...v0.3.1)
 
@@ -62,16 +62,14 @@ SPDX-License-Identifier: AGPL-3.0-or-later
 
 ## [v0.2.0](https://github.com/rero/rero-invenio-base/tree/v0.2.0) (2023-01-31)
 
-[Full Changelog](https://github.com/rero/rero-invenio-base/compare/...v0.2.0)
+[Full Changelog](https://github.com/rero/rero-invenio-base/compare/v0.1.0...v0.2.0)
 
 **Changes:**
 
 * task: add a generic celery task [\#10](https://github.com/rero/rero-invenio-base/pull/10) (by @jma)
 
 ## [v0.1.0](https://github.com/rero/rero-invenio-base/tree/v0.1.0) (2022-09-02)
 
-[Full Changelog](https://github.com/rero/rero-invenio-base/compare/...v0.1.0)
-
 **Changes:**
 
 * cli: adds snapshot lifecycle management commands [\#6](https://github.com/rero/rero-invenio-base/pull/6) (by @rerowep)

CLAUDE.md

@@ -0,0 +1,88 @@
+# rero-invenio-base Claude guide
+
+## Overview
+
+rero-invenio-base is a Python library providing generic backend utilities for RERO Invenio instances. It ships SEARCH CLI management commands, a streaming data export framework, and shared Celery tasks.
+
+**Stack**: Python 3.12–3.14, Flask (Invenio), ElasticSearch 7, Celery
+**Package manager**: `uv` with `poethepoet` for task running
+
+## Commands
+
+All commands are run through uv's virtual env with `uv run`.
+
+### Linting and formatting
+
+**IMPORTANT:** After editing files, make sure that there are no errors in the formatting and linting.
+
+```bash
+uv run poe lint     # ruff check
+uv run poe format   # ruff format
+```
+
+### Setup (done by humans)
+
+Human developers will start the required containers and services on their own terms. Tests require SEARCH to be running.
+
+## Architecture
+
+### Package structure
+
+```text
+rero_invenio_base/
+├── ext.py                    # Flask extension (REROInvenioBase)
+├── config.py                 # Default configuration keys
+├── cli/
+│   ├── shared.py             # Shared CLI helpers (abort_if_false)
+│   ├── utils.py              # check_license, check_json commands
+│   └── es/
+│       ├── alias.py          # `rero es alias` commands
+│       ├── index.py          # `rero es index` commands (reindex, move, update-mapping, …)
+│       ├── task.py           # `rero es task` commands
+│       ├── snapshot/         # `rero es snapshot` commands
+│       └── slm/              # `rero es slm` snapshot-management commands
+└── modules/
+    ├── tasks.py              # run_on_worker Celery task
+    ├── utils.py              # chunk() utility
+    └── export/
+        ├── ext.py            # ReroInvenioBaseExportApp Flask extension
+        ├── views.py          # ExportResource view + create_blueprint_from_app
+        └── config.py         # Export REST endpoint configuration
+```
+
+### Entry points
+
+Registered in `pyproject.toml`:
+
+| Entry point group | Name | Target |
+|---|---|---|
+| `flask.commands` | `rero` | `rero_invenio_base.cli:rero` |
+| `invenio_base.apps` | `rero-invenio-base-export` | `ReroInvenioBaseExportApp` |
+| `invenio_base.api_blueprints` | `rero_ils_exports` | `create_blueprint_from_app` |
+| `invenio_celery.tasks` | `rero` | `rero_invenio_base.modules.tasks` |
+
+### Export module
+
+`ExportResource` is a `ContentNegotiatedMethodView` that streams record search results. Routes are registered dynamically from the `RERO_INVENIO_BASE_EXPORT_REST_ENDPOINTS` config key via `create_blueprint_from_app`. Each endpoint maps MIME types to serializers and prepends `/export` to the list route.
+
+## Code style
+
+- Be clear and concise in docstrings; do not over-comment the code.
+- Do not use Python type annotations (no `-> str`, `: str`, etc. in signatures).
+- Use **Sphinx-style** docstrings (`:param x:`, `:returns:`, `:rtype:`).
+- Commit messages follow [Conventional Commits](https://www.conventionalcommits.org).
+
+## Testing
+
+- Tests use function-based style (no class-based tests).
+- Test fixtures are in `tests/conftest.py`; sample data in `tests/data/`.
+- `--doctest-modules` is active — doctests in module files are run by default.
+
+### Running the tests (done by humans)
+
+Human developers run tests from their consoles after starting the SEARCH container:
+
+```bash
+uv run ./scripts/test   # full suite (lint + pip-audit + pytest with SEARCH)
+uv run pytest           # pytest only (SEARCH must already be running)
+```

pyproject.toml

@@ -39,7 +39,6 @@ dev = [
 
 [tool.project.entry-points]
 check_json = "rero_invenio_base.cli.utils:check_json"
-rero = "rero_invenio_base.cli:rero"
 
 [project.entry-points."flask.commands"]
 rero = "rero_invenio_base.cli:rero"

rero_invenio_base/cli/__init__.py

@@ -17,4 +17,4 @@ def rero():
 rero.add_command(utils)
 rero.add_command(es)
 
-__all__ = "rero"
+__all__ = ["rero"]

rero_invenio_base/cli/es/__init__.py

@@ -1,26 +1,30 @@
 # SPDX-FileCopyrightText: Fondation RERO+
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
-"""Click elasticsearch command-line utilities."""
+"""Click Search command-line utilities."""
 
 import click
 
 from .alias import alias
+from .health import health
 from .index import index
+from .queue import queue
 from .slm import slm
 from .snapshot import snapshot
 from .task import task
 
 
 @click.group()
 def es():
-    """Elasticsarch management commands."""
+    """Search management commands."""
 
 
-es.add_command(index)
 es.add_command(alias)
+es.add_command(health)
+es.add_command(index)
+es.add_command(queue)
 es.add_command(slm)
 es.add_command(snapshot)
 es.add_command(task)
 
-__all__ = "index"
+__all__ = ["es"]

rero_invenio_base/cli/es/alias.py

@@ -1,7 +1,7 @@
 # SPDX-FileCopyrightText: Fondation RERO+
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
-"""Click elasticsearch index command-line utilities."""
+"""Click Search index command-line utilities."""
 
 import json
 
@@ -14,13 +14,13 @@
 
 @click.group()
 def alias():
-    """Elasticsearch alias commands."""
+    """Search alias commands."""
 
 
 @alias.command("get")
 @with_appcontext
 def get_alias():
-    """Get elasticsearch aliases."""
+    """Get Search aliases."""
     click.secho(json.dumps(current_search_client.indices.get_alias(), indent=2), fg="green")
 
 
@@ -29,7 +29,7 @@ def get_alias():
 @click.argument("index")
 @click.argument("name")
 def put_alias(index, name):
-    """Put elasticsearch alias."""
+    """Put Search alias."""
     try:
         click.secho(
             json.dumps(current_search_client.indices.put_alias(index, name), indent=2),
@@ -51,7 +51,7 @@ def put_alias(index, name):
     prompt="Do you really want to delete an alias?",
 )
 def delete_alias(index, name):
-    """Delete elasticsearch alias."""
+    """Delete Search alias."""
     try:
         click.secho(
             json.dumps(current_search_client.indices.delete_alias(index, name), indent=2),

rero_invenio_base/cli/es/health.py

@@ -0,0 +1,75 @@
+# RERO Invenio Base
+# Copyright (C) 2025 RERO.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+"""Click health-check command-line utility."""
+
+from urllib.parse import urlparse, urlunparse
+
+import click
+from flask import current_app
+from flask.cli import with_appcontext
+from invenio_db import db
+from invenio_search import current_search_client
+from sqlalchemy import text
+
+
+@click.command("health")
+@with_appcontext
+def health():
+    """Check connectivity to SEARCH, database, and Redis.
+
+    Exits with status 1 if any service is unreachable or in error state.
+    """
+    all_ok = True
+
+    # SEARCH
+    try:
+        s = current_search_client.cluster.health()
+        color = {"green": "green", "yellow": "yellow"}.get(s["status"], "red")
+        label = "OK" if s["status"] != "red" else "ERROR"
+        detail = f"{s['status'].upper()} — {s['number_of_nodes']} node(s), cluster: {s['cluster_name']}"
+        click.secho(f"{'SEARCH':<12} {label:<7} {detail}", fg=color)
+        if s["status"] == "red":
+            all_ok = False
+    except Exception as err:
+        click.secho(f"{'SEARCH':<12} {'ERROR':<7} {err}", fg="red")
+        all_ok = False
+
+    # Database
+    try:
+        db.session.execute(text("SELECT 1"))
+        click.secho(f"{'Database':<12} {'OK':<7}", fg="green")
+    except Exception as err:
+        click.secho(f"{'Database':<12} {'ERROR':<7} {err}", fg="red")
+        all_ok = False
+
+    # Broker / Redis — use Celery's own connection so any broker type works.
+    try:
+        from celery import current_app as celery_app
+
+        with celery_app.connection() as conn:
+            conn.ensure_connection(max_retries=1)
+        broker_url = current_app.config.get("BROKER_URL") or current_app.config.get("CELERY_BROKER_URL", "")
+        parsed = urlparse(broker_url)
+        host = parsed.hostname or ""
+        netloc = f"{host}:{parsed.port}" if parsed.port else host
+        safe_url = urlunparse(parsed._replace(netloc=netloc))
+        click.secho(f"{'Broker':<12} {'OK':<7} {safe_url[:60]}", fg="green")
+    except Exception as err:
+        click.secho(f"{'Broker':<12} {'ERROR':<7} {err}", fg="red")
+        all_ok = False
+
+    if not all_ok:
+        raise SystemExit(1)