Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions ingestion/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ All notable changes to `zep-ingest` are documented here. The project follows
[Semantic Versioning](https://semver.org); while at `0.x` the public API may
still change between minor versions.

## Unreleased

- New named [OrcaRouter](https://www.orcarouter.ai) adapter `OrcaRouterLLM`
(`zep_ingest.llm.orcarouter`): pre-fills the OpenAI-compatible gateway
`https://api.orcarouter.ai/v1` and the adaptive `orcarouter/auto` model, and
reads `ORCAROUTER_API_KEY` (prefix `sk-orca-`).

## 0.2.1

- Batch submission now rolls over at 10,000 items by default
Expand Down
25 changes: 25 additions & 0 deletions ingestion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ own engine — uses for its LLM clients):
```python
from zep_ingest.llm.openai import OpenAILLM, OpenAICompatibleLLM
from zep_ingest.llm.anthropic import AnthropicLLM
from zep_ingest.llm.orcarouter import OrcaRouterLLM

OpenAILLM() # OpenAI (pip install "zep-ingest[openai]")
AnthropicLLM() # Anthropic (pip install "zep-ingest[anthropic]")
Expand All @@ -235,12 +236,36 @@ OpenAICompatibleLLM(
base_url="http://localhost:11434/v1", # OpenRouter, Together, ...
api_key="ollama",
)
# Named gateway adapter — OrcaRouter (https://www.orcarouter.ai):
OrcaRouterLLM() # reads ORCAROUTER_API_KEY, defaults to orcarouter/auto
```

`OpenAICompatibleLLM` is the docs-recommended route to "any provider":
LiteLLM alone proxies 100+ models behind this interface, with no extra
dependency here beyond the `openai` package.

[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible gateway that
routes to the best model for each request. `OrcaRouterLLM` pre-fills its
`https://api.orcarouter.ai/v1` base URL and the adaptive `orcarouter/auto`
model, and reads the API key from `ORCAROUTER_API_KEY` (prefix `sk-orca-`),
so contextualization needs no endpoint/model plumbing:

```python
from zep_ingest import LLMContextualizer, ingest_documents
from zep_ingest.llm.orcarouter import OrcaRouterLLM

ingest_documents(
client,
"handbook/**/*.md",
graph_id="company_kb",
llm=LLMContextualizer(OrcaRouterLLM()),
)
```

It also runs gateway-level, zero-trust security for AI agents on the same
endpoint — screening every prompt/response and governing every tool call on a
default-deny basis, with no application code changes.

## Entity canonicalization

Zep merges entities by the names it sees in text; semantic aliases ("PROTOTYPE-202" vs
Expand Down
11 changes: 11 additions & 0 deletions ingestion/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,17 @@ pip install zep-ingest
pip install "zep-ingest[anthropic]" # and/or "zep-ingest[openai]"
```

For [OrcaRouter](https://www.orcarouter.ai) contextualization (the named
`OrcaRouterLLM` adapter), grab an API key from the [OrcaRouter
dashboard](https://www.orcarouter.ai) (prefix `sk-orca-`) and export it:

```bash
export ORCAROUTER_API_KEY="sk-orca-..."
```

The adapter defaults to `https://api.orcarouter.ai/v1` with the adaptive
`orcarouter/auto` model; see the README "Bring any LLM" section.

Requires Python ≥ 3.11.

## 3. Batch API vs sequential
Expand Down
7 changes: 7 additions & 0 deletions ingestion/src/zep_ingest/llm/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"""LLM adapters for the :class:`zep_ingest.protocols.LLMClient` protocol."""

from zep_ingest.llm.anthropic import AnthropicLLM
from zep_ingest.llm.openai import OpenAICompatibleLLM, OpenAILLM
from zep_ingest.llm.orcarouter import OrcaRouterLLM

__all__ = ["AnthropicLLM", "OpenAILLM", "OpenAICompatibleLLM", "OrcaRouterLLM"]
2 changes: 2 additions & 0 deletions ingestion/src/zep_ingest/llm/openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
which is the pattern Zep's own docs recommend for bring-your-own-provider
setups. Both are optional conveniences: anything implementing
``complete(prompt) -> str`` satisfies the LLMClient protocol directly.
For a named OrcaRouter gateway adapter (with a pre-filled base URL and
model), see :mod:`zep_ingest.llm.orcarouter`.

Optional dependency: pip install "zep-ingest[openai]" (not needed when you
pass a pre-constructed client).
Expand Down
54 changes: 54 additions & 0 deletions ingestion/src/zep_ingest/llm/orcarouter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""OrcaRouter adapter for the LLMClient protocol (optional: pip install zep-ingest[openai]).

[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible gateway that
routes to the best model for each request. It also runs gateway-level,
zero-trust security for AI agents on the same endpoint — screening every
prompt/response and governing every tool call on a default-deny basis, with
no application code changes.

``OrcaRouterLLM`` is a named convenience built on ``OpenAICompatibleLLM``:
it pre-fills the gateway base URL and the adaptive ``orcarouter/auto`` model
so a contextualizer can be wired up in one line:

from zep_ingest.llm.orcarouter import OrcaRouterLLM

LLMContextualizer(OrcaRouterLLM()) # reads ORCAROUTER_API_KEY

The API key is read from the ``ORCAROUTER_API_KEY`` environment variable
(prefix ``sk-orca-``) unless one is passed explicitly.
"""

from typing import Any

from zep_ingest.llm.openai import OpenAICompatibleLLM

DEFAULT_ORCAROUTER_BASE_URL = "https://api.orcarouter.ai/v1"
DEFAULT_ORCAROUTER_MODEL = "orcarouter/auto"


class OrcaRouterLLM(OpenAICompatibleLLM):
"""Named adapter for OrcaRouter's OpenAI-compatible gateway.

Defaults to ``https://api.orcarouter.ai/v1`` with the adaptive
``orcarouter/auto`` model. The API key defaults to the
``ORCAROUTER_API_KEY`` environment variable.
"""

def __init__(
self,
*,
model: str = DEFAULT_ORCAROUTER_MODEL,
base_url: str = DEFAULT_ORCAROUTER_BASE_URL,
api_key: str | None = None,
client: Any | None = None,
max_tokens: int = 200,
) -> None:
import os

super().__init__(
model=model,
base_url=base_url,
api_key=os.environ.get("ORCAROUTER_API_KEY") if api_key is None else api_key,
client=client,
max_tokens=max_tokens,
)
65 changes: 65 additions & 0 deletions ingestion/tests/test_llm_adapters.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,3 +114,68 @@ def test_missing_sdk_raises_dependency_error(self, monkeypatch):
monkeypatch.setitem(sys.modules, "openai", None)
with pytest.raises(ZepDependencyError, match="zep-ingest\\[openai\\]"):
OpenAICompatibleLLM(model="m", base_url="http://localhost:4000")


class TestOrcaRouterAdapter:
"""Named adapter for OrcaRouter's OpenAI-compatible gateway."""

def test_defaults_to_orcarouter_base_url_and_model(self, monkeypatch):
import sys

from zep_ingest.llm.orcarouter import OrcaRouterLLM

fake_openai = MagicMock()
monkeypatch.setitem(sys.modules, "openai", fake_openai)
monkeypatch.setenv("ORCAROUTER_API_KEY", "sk-orca-test")
llm = OrcaRouterLLM()
fake_openai.OpenAI.assert_called_once_with(
base_url="https://api.orcarouter.ai/v1", api_key="sk-orca-test"
)
assert llm.model == "orcarouter/auto"

def test_env_key_used_when_api_key_not_given(self, monkeypatch):
import sys

from zep_ingest.llm.orcarouter import OrcaRouterLLM

fake_openai = MagicMock()
monkeypatch.setitem(sys.modules, "openai", fake_openai)
monkeypatch.setenv("ORCAROUTER_API_KEY", "sk-orca-env")
OrcaRouterLLM()
fake_openai.OpenAI.assert_called_once_with(
base_url="https://api.orcarouter.ai/v1", api_key="sk-orca-env"
)

def test_explicit_api_key_overrides_env(self, monkeypatch):
import sys

from zep_ingest.llm.orcarouter import OrcaRouterLLM

fake_openai = MagicMock()
monkeypatch.setitem(sys.modules, "openai", fake_openai)
monkeypatch.setenv("ORCAROUTER_API_KEY", "sk-orca-env")
OrcaRouterLLM(api_key="sk-orca-explicit")
fake_openai.OpenAI.assert_called_once_with(
base_url="https://api.orcarouter.ai/v1", api_key="sk-orca-explicit"
)

def test_complete_uses_chat_completions(self):
from zep_ingest.llm.orcarouter import OrcaRouterLLM

client = MagicMock()
client.chat.completions.create.return_value = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="the context"))]
)
llm = OrcaRouterLLM(client=client)
assert llm.complete("prompt text") == "the context"
kwargs = client.chat.completions.create.call_args.kwargs
assert kwargs["model"] == "orcarouter/auto"

def test_missing_sdk_raises_dependency_error(self, monkeypatch):
import sys

from zep_ingest.llm.orcarouter import OrcaRouterLLM

monkeypatch.setitem(sys.modules, "openai", None)
with pytest.raises(ZepDependencyError, match="zep-ingest\\[openai\\]"):
OrcaRouterLLM()