feat(ingestion): Zep document_id grouping for zep-ingest 0.4.0

.github/workflows/smoke-ingestion.yml

@@ -0,0 +1,48 @@
+name: Smoke Ingestion Package
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'ingestion/**'
+      - '.github/workflows/smoke-ingestion.yml'
+  workflow_run:
+    workflows: [Release Ingestion Package]
+    types: [completed]
+
+permissions:
+  contents: read
+
+jobs:
+  smoke:
+    if: >-
+      github.event_name == 'push' ||
+      (github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success')
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    environment: integration
+    defaults:
+      run:
+        working-directory: ingestion
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          # workflow_run otherwise checks out current default-branch HEAD, not
+          # the commit the Release Ingestion Package workflow ran on.
+          ref: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_sha || github.sha }}
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3
+        with:
+          version: "0.7.19"
+
+      - name: Set up Python
+        run: uv python install 3.13
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Run live smoke tests
+        run: uv run python scripts/smoke.py
+        env:
+          ZEP_API_KEY: ${{ secrets.ZEP_API_KEY }}

.github/workflows/test-ingestion.yml

@@ -61,7 +61,7 @@ jobs:
           name: ingestion-coverage
 
   integration:
-    if: github.event_name == 'push'
+    if: false  # superseded by smoke-ingestion.yml (non-blocking post-merge smoke)
     runs-on: ubuntu-latest
     environment: integration
     defaults:

ingestion/CHANGELOG.md

@@ -4,6 +4,24 @@ All notable changes to `zep-ingest` are documented here. The project follows
 [Semantic Versioning](https://semver.org); while at `0.x` the public API may
 still change between minor versions.
 
+## 0.4.0
+
+- **Document grouping (`document_id`).** Text files, transcripts, and emails
+  always get a `document_id` (even as a single episode). Slack is ingested as
+  one episode per message with a shared `document_id` per Slack thread, plus a
+  second document of each channel's top-level messages in order (thread parents
+  are ingested twice; payload identical except `document_id`). Not a Zep
+  `thread_id`. The Slack `grouping` option is removed. JSON records omit
+  `document_id`. `TextChunker` preserves `document_id` across chunks.
+- Sequential `graph.add` sends `document_id` via a thin API wrapper until the
+  installed `zep-cloud` SDK exposes the field. Batch items pass it through
+  `BatchAddItem` extras.
+- `wait()` polls one tail episode per `document_id` saga (like multi-thread
+  polling). Ingestion order within a document follows `created_at`.
+- Unified live smoke suite: `ingestion/scripts/smoke.py` (also
+  `pytest -m smoke`). Runs post-merge and post-release via
+  `.github/workflows/smoke-ingestion.yml` with `continue-on-error: true`.
+
 ## 0.3.0
 
 - **Submit everything, then wait once.** Multiple files or loaders destined for

ingestion/examples/README.md

@@ -24,7 +24,7 @@ the [dashboard](https://app.getzep.com) when you're done.
 | [`documents_example.py`](documents_example.py) | Markdown → ~500-char chunks; optional LLM contextualization (auto-detects `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`) | named graph |
 | [`fact_triples_example.py`](fact_triples_example.py) | molding a realistic directory export (no triple-shaped columns) into explicit fact triples; the manual create → set_ontology → seed lifecycle | named graph |
 | [`json_records_example.py`](json_records_example.py) | structured records with identity-field mapping — Zep extracts the relationships | named graph |
-| [`slack_export_example.py`](slack_export_example.py) | free `preview()` first, then a Slack export with thread grouping, `skip_subtypes`, and the opt-in risky-alias guard | named graph |
+| [`slack_export_example.py`](slack_export_example.py) | free `preview()` first, then a Slack export (one episode per message, thread as `document_id`), `skip_subtypes`, and the opt-in risky-alias guard | named graph |
 | [`user_graph_example.py`](user_graph_example.py) | **combined**: profile fact triples → chat-thread backfill → a document, all on one user's graph | user graph |
 | [`thread_backfill_example.py`](thread_backfill_example.py) | historic chat history (JSONL) → threads that power `thread.get_user_context()` | user graph |
 

ingestion/examples/slack_export_example.py

@@ -2,8 +2,9 @@
 
 Self-contained and re-runnable: previews then ingests the bundled sample
 export under examples/data/slack_export/, creating a fresh graph and setting
-the starter ontology first. Messages are grouped by thread, join/leave noise
-is skipped, and each episode keeps its real Slack timestamp.
+the starter ontology first. Each message is one episode; Slack threads share a
+``document_id``. Join/leave noise is skipped, and each episode keeps its real
+Slack timestamp.
 
 To run against real data, get an export from Slack (Workspace Admin →
 Settings & administration → Workspace settings → Import/Export Data →

ingestion/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "zep-ingest"
-version = "0.3.0"
+version = "0.4.0"
 description = "Bulk data ingestion pipeline for Zep: chunk, contextualize, canonicalize, and submit unstructured and structured data"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -57,7 +57,8 @@ python_files = ["test_*.py"]
 python_classes = ["Test*"]
 python_functions = ["test_*"]
 markers = [
-    "integration: marks tests as integration tests (deselect with '-m \"not integration\"')"
+    "integration: marks tests as integration tests (deselect with '-m \"not integration\"')",
+    "smoke: live smoke tests against a real Zep account",
 ]
 
 [tool.ruff]