Skip to content

Commit 33e3bfd

Browse files
varun29ankuSvarun29ankuS
committed
feat: add validate_deep, demo script, and rewrite README as landing page
- validate_deep: checks UUID validity/uniqueness, reference integrity, timestamps, embedding dimensions, graph entity/relationship refs - demo.py: end-to-end conversion example (mem0 → MIF → markdown → back) - README: rewritten as developer-facing landing page with install, CLI examples, Python API, MCP integration, format table
1 parent d60713b commit 33e3bfd

4 files changed

Lines changed: 638 additions & 55 deletions

File tree

‎README.md‎

Lines changed: 111 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,19 @@
11
# Memory Interchange Format (MIF)
22

3-
A vendor-neutral JSON schema for portable AI agent memories.
3+
**Your AI agent has 6 months of memories in System A. You want to try System B. Without MIF, you lose everything. With MIF:**
44

5-
## The Problem
6-
7-
30+ AI memory systems exist — mem0, Zep, Cognee, Letta, basic-memory, shodh-memory, and more. Each stores fundamentally the same data (id, content, timestamp, type, metadata) in incompatible formats. There is no way to move memories between systems.
8-
9-
**MIF solves memory portability.** Like vCard for contacts or iCalendar for events — a minimal envelope so memories can move between providers.
10-
11-
## What MIF Is
12-
13-
- A JSON schema for exporting and importing AI agent memories
14-
- A minimal core: `id`, `content`, `created_at` — that's it for a valid memory
15-
- Optional knowledge graph support for systems that track entity relationships
16-
- Vendor extensions so systems preserve proprietary metadata without polluting the core
17-
- Privacy/PII redaction built into the export metadata
5+
```bash
6+
pip install mif-tools
7+
mif convert mem0_export.json --to shodh -o memories.mif.json
8+
```
189

19-
## What MIF Is Not
10+
Done. Your memories are portable.
2011

21-
- Not a specification for how memory systems work internally
22-
- Not a database format or storage engine
23-
- Not opinionated about retrieval, embeddings, or ranking strategies
12+
## What is MIF?
2413

25-
## Quick Example
14+
A vendor-neutral JSON envelope for AI agent memories. Like vCard for contacts or iCalendar for events — a minimal schema so memories move between providers without data loss.
2615

27-
A minimal conforming MIF document:
16+
**3 required fields.** That's it.
2817

2918
```json
3019
{
@@ -39,64 +28,132 @@ A minimal conforming MIF document:
3928
}
4029
```
4130

42-
A full document can include memory types, entities, embeddings, knowledge graph, vendor extensions, and privacy metadata. See [examples/](./examples/) for complete samples.
31+
Everything else — memory types, tags, entities, embeddings, knowledge graph, vendor extensions — is optional. Add what you have, ignore what you don't.
32+
33+
## Install
34+
35+
```bash
36+
pip install mif-tools # core (zero dependencies)
37+
pip install mif-tools[validate] # with JSON Schema validation
38+
```
39+
40+
## Convert Between Formats
41+
42+
```bash
43+
# mem0 → MIF
44+
mif convert mem0_export.json --from mem0 -o memories.mif.json
45+
46+
# MIF → Markdown (Obsidian/Letta style)
47+
mif convert memories.mif.json --to markdown -o memories.md
4348

44-
## Specification
49+
# Auto-detect source format
50+
mif convert any_memory_file.json -o output.mif.json
4551

46-
The full specification is in [`spec/mif-v2.md`](./spec/mif-v2.md).
52+
# Inspect any memory file
53+
mif inspect memories.json
4754

48-
Key sections:
49-
- **Memory Object** — required and optional fields
50-
- **Knowledge Graph** — optional entity and relationship data
51-
- **Vendor Extensions** — system-specific metadata preservation
52-
- **Privacy** — PII detection and redaction
53-
- **Import Behavior** — UUID preservation, deduplication, partial failure tolerance
55+
# Validate MIF document
56+
mif validate memories.mif.json
57+
```
5458

55-
## JSON Schema
59+
## Python API
60+
61+
```python
62+
from mif import load, dump, convert, MifDocument, Memory
63+
64+
# Load from any format (auto-detects mem0, markdown, generic JSON, MIF)
65+
doc = load(open("mem0_export.json").read())
66+
print(f"{len(doc.memories)} memories loaded")
67+
68+
# Convert between formats in one line
69+
markdown = convert(data, from_format="mem0", to_format="markdown")
70+
71+
# Create memories from scratch
72+
doc = MifDocument(memories=[
73+
Memory(
74+
id="123e4567-e89b-12d3-a456-426614174000",
75+
content="User prefers dark mode",
76+
created_at="2026-01-15T10:30:00Z",
77+
memory_type="observation",
78+
tags=["preferences", "ui"],
79+
)
80+
])
81+
print(dump(doc)) # MIF v2 JSON
82+
83+
# Deep validation (UUIDs, references, timestamps, embedding dimensions)
84+
from mif import validate_deep
85+
ok, warnings = validate_deep(open("export.mif.json").read())
86+
```
5687

57-
Validate MIF documents against [`schema/mif-v2.schema.json`](./schema/mif-v2.schema.json).
88+
## Add MIF to Your MCP Server (10 lines)
5889

59-
## Adapters
90+
```python
91+
from mif import load, dump
6092

61-
Format adapters bridge existing memory systems to MIF:
93+
# Export handler
94+
def export_memories(user_id: str) -> str:
95+
memories = my_storage.get_all(user_id)
96+
return dump(memories)
6297

63-
| System | Status | Location |
64-
|--------|--------|----------|
65-
| [shodh-memory](https://github.com/varun29ankuS/shodh-memory) | Production | Built-in (`/api/export/mif`, `/api/import/mif`) |
66-
| mem0 JSON | Production | [Rust adapter](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/mem0.rs) |
67-
| Markdown (YAML frontmatter) | Production | [Rust adapter](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/markdown.rs) |
68-
| Generic JSON | Production | [Rust adapter](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/generic.rs) |
98+
# Import handler — auto-detects mem0, markdown, generic JSON, MIF
99+
def import_memories(data: str) -> dict:
100+
doc = load(data)
101+
for mem in doc.memories:
102+
my_storage.save(mem.id, mem.content, mem.created_at)
103+
return {"memories_imported": len(doc.memories)}
104+
```
105+
106+
## Supported Formats
107+
108+
| Format | ID | Auto-detect | Description |
109+
|--------|----|-------------|-------------|
110+
| **MIF v2** | `shodh` | `"mif_version"` in JSON | Native format, lossless round-trip |
111+
| **mem0** | `mem0` | JSON array with `"memory"` field | mem0 memory exports |
112+
| **Generic JSON** | `generic` | JSON array with `"content"` field | Any JSON memory array |
113+
| **Markdown** | `markdown` | Starts with `---` | YAML frontmatter (Letta/Obsidian style) |
114+
115+
## Full Spec
116+
117+
MIF supports optional fields for rich memory data:
118+
119+
- **Memory types** — `observation`, `decision`, `learning`, `error`, `context`, `conversation`, and custom types
120+
- **Entity references** — named entities with type and confidence
121+
- **Embeddings** — model name, dimensions, vector (reuse or regenerate)
122+
- **Knowledge graph** — entities and relationships with confidence scores
123+
- **Vendor extensions** — system-specific metadata preserved on round-trip
124+
- **Privacy** — PII detection and redaction markers
125+
126+
Full specification: [`spec/mif-v2.md`](./spec/mif-v2.md) | JSON Schema: [`schema/mif-v2.schema.json`](./schema/mif-v2.schema.json)
127+
128+
## Adapters & Implementations
129+
130+
| System | Status | Type |
131+
|--------|--------|------|
132+
| [shodh-memory](https://github.com/varun29ankuS/shodh-memory) | Production | Built-in HTTP API (`/api/export/mif`, `/api/import/mif`) |
133+
| [mif-tools](https://pypi.org/project/mif-tools/) | Production | Python package with CLI |
134+
| mem0 | Adapter ready | Python + [Rust](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/mem0.rs) |
135+
| Markdown (YAML frontmatter) | Adapter ready | Python + [Rust](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/markdown.rs) |
136+
| Generic JSON | Adapter ready | Python + [Rust](https://github.com/varun29ankuS/shodh-memory/blob/main/src/mif/adapters/generic.rs) |
69137
| CrewAI | Planned | — |
70138
| LangChain | Planned | — |
71139

72140
## Design Principles
73141

74-
1. **Minimal** — Only memories + optional graph. No todos, projects, or other concerns.
142+
1. **Minimal** — 3 required fields. Everything else is optional.
75143
2. **Extensible** — Unknown fields and vendor extensions MUST be preserved on round-trip.
76144
3. **Vendor-neutral** — The schema doesn't favor any implementation.
77145
4. **Forward-compatible** — Importers MUST ignore unknown fields.
78146

79147
## Contributing
80148

81-
We welcome adapter implementations for any memory system. See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines.
149+
We welcome adapter implementations for any memory system. See [CONTRIBUTING.md](./CONTRIBUTING.md).
82150

83151
## Related
84152

85153
- [MCP SEP #2342](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2342) — Original proposal to the Model Context Protocol
86154
- [tower-mcp #531](https://github.com/joshrotenberg/tower-mcp/issues/531) — Tracking issue in tower-mcp
87-
- [shodh-memory](https://github.com/varun29ankuS/shodh-memory) — Reference implementation
88-
89-
## Validation
90-
91-
```bash
92-
pip install -r requirements.txt
93-
python validate.py examples/*.mif.json
94-
python tests/round_trip.py examples/*.mif.json
95-
```
96-
97-
## Version Note
98-
99-
MIF v2.0 is the first public release. The "v2" numbering reflects internal iterations during development in shodh-memory. There is no public v1 specification.
155+
- [shodh-memory](https://github.com/varun29ankuS/shodh-memory) — Reference implementation (Rust)
156+
- [mif-tools on PyPI](https://pypi.org/project/mif-tools/) — Python package
100157

101158
## License
102159

0 commit comments

Comments
 (0)