|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +ProllyTree is a probabilistic tree data structure that combines B-trees and Merkle trees, implemented in Rust with Python bindings. It provides efficient data access with cryptographic verification, designed for distributed systems, version control, and AI memory systems. |
| 8 | + |
| 9 | +## Core Architecture |
| 10 | + |
| 11 | +### Language & Framework |
| 12 | +- **Primary Language**: Rust (edition 2021) |
| 13 | +- **Python Bindings**: Available via PyO3 (Python 3.8+) |
| 14 | +- **Binary**: `git-prolly` CLI tool for Git-like versioned key-value storage |
| 15 | + |
| 16 | +### Key Components |
| 17 | +- **Core Tree**: `src/tree.rs` - Probabilistic B-tree implementation with Merkle hashing |
| 18 | +- **Storage Backends**: In-memory, RocksDB, and Git-backed storage options |
| 19 | +- **Git Integration**: `src/git/` - Git-like version control for key-value data |
| 20 | +- **SQL Support**: `src/sql.rs` - GlueSQL integration for SQL queries on tree data |
| 21 | +- **Agent Memory**: `src/agent/` - AI agent memory system with semantic, episodic, and working memory |
| 22 | +- **Python Module**: `src/python.rs` - PyO3 bindings for Python integration |
| 23 | + |
| 24 | +### Feature Flags |
| 25 | +- `git`: Git-backed versioned storage |
| 26 | +- `sql`: SQL query support via GlueSQL |
| 27 | +- `rig`: Rig framework integration for AI agents |
| 28 | +- `python`: Python bindings |
| 29 | +- `rocksdb_storage`: RocksDB persistent storage |
| 30 | +- `tui`: Terminal UI for interactive usage |
| 31 | + |
| 32 | +## Common Commands |
| 33 | + |
| 34 | +### Build & Development |
| 35 | +```bash |
| 36 | +# Build the project |
| 37 | +cargo build |
| 38 | + |
| 39 | +# Build with all features |
| 40 | +cargo build --all-features |
| 41 | + |
| 42 | +# Build release version with optimizations |
| 43 | +cargo build --release |
| 44 | + |
| 45 | +# Build specific features |
| 46 | +cargo build --features "git sql" |
| 47 | + |
| 48 | +# Build the git-prolly CLI tool |
| 49 | +cargo build --features "git sql" --bin git-prolly |
| 50 | +``` |
| 51 | + |
| 52 | +### Testing |
| 53 | +```bash |
| 54 | +# Run all tests |
| 55 | +cargo test |
| 56 | + |
| 57 | +# Run specific test |
| 58 | +cargo test test_name |
| 59 | + |
| 60 | +# Run tests with output |
| 61 | +cargo test -- --nocapture |
| 62 | + |
| 63 | +# Run tests for specific module |
| 64 | +cargo test --lib tree::tests |
| 65 | + |
| 66 | +# Run with specific features |
| 67 | +cargo test --features "git sql" |
| 68 | +``` |
| 69 | + |
| 70 | +### Code Quality |
| 71 | +```bash |
| 72 | +# Format code |
| 73 | +cargo fmt |
| 74 | + |
| 75 | +# Check formatting without changes |
| 76 | +cargo fmt -- --check |
| 77 | + |
| 78 | +# Run linter |
| 79 | +cargo clippy --all |
| 80 | + |
| 81 | +# Check code without building |
| 82 | +cargo check |
| 83 | + |
| 84 | +# Generate documentation |
| 85 | +cargo doc --document-private-items --no-deps |
| 86 | +``` |
| 87 | + |
| 88 | +### Python Development |
| 89 | +```bash |
| 90 | +# Build Python bindings |
| 91 | +./python/build_python.sh |
| 92 | + |
| 93 | +# Build and install Python bindings |
| 94 | +./python/build_python.sh --install |
| 95 | + |
| 96 | +# Run Python tests (after building) |
| 97 | +python -m pytest python/tests/ |
| 98 | + |
| 99 | +# Run Python examples |
| 100 | +cd python/examples && ./run_examples.sh |
| 101 | +``` |
| 102 | + |
| 103 | +### Git-Prolly CLI Usage |
| 104 | +```bash |
| 105 | +# Initialize a new repository |
| 106 | +./target/debug/git-prolly init |
| 107 | + |
| 108 | +# Set key-value pairs |
| 109 | +./target/debug/git-prolly set key1 value1 |
| 110 | +./target/debug/git-prolly set key2 value2 |
| 111 | + |
| 112 | +# Commit changes |
| 113 | +./target/debug/git-prolly commit -m "Initial data" |
| 114 | + |
| 115 | +# List all keys |
| 116 | +./target/debug/git-prolly list |
| 117 | +./target/debug/git-prolly list --values # Include values |
| 118 | +./target/debug/git-prolly list --graph # Show tree structure |
| 119 | + |
| 120 | +# Get specific value |
| 121 | +./target/debug/git-prolly get key1 |
| 122 | + |
| 123 | +# View commit history |
| 124 | +./target/debug/git-prolly log |
| 125 | +./target/debug/git-prolly log --limit 5 |
| 126 | + |
| 127 | +# SQL queries |
| 128 | +./target/debug/git-prolly sql "CREATE TABLE users (id INTEGER, name TEXT)" |
| 129 | +./target/debug/git-prolly sql "INSERT INTO users VALUES (1, 'Alice')" |
| 130 | +./target/debug/git-prolly sql "SELECT * FROM users" |
| 131 | +``` |
| 132 | + |
| 133 | +### Benchmarking |
| 134 | +```bash |
| 135 | +# Run tree benchmarks |
| 136 | +cargo bench --bench tree |
| 137 | + |
| 138 | +# Run SQL benchmarks |
| 139 | +cargo bench --bench sql |
| 140 | + |
| 141 | +# Run Git benchmarks |
| 142 | +cargo bench --bench git |
| 143 | +``` |
| 144 | + |
| 145 | +## Testing Patterns |
| 146 | + |
| 147 | +### Rust Tests |
| 148 | +- Unit tests are in the same file as the code using `#[cfg(test)]` modules |
| 149 | +- Integration tests would go in `tests/` directory (currently not present) |
| 150 | +- Use `RUST_BACKTRACE=1` for debugging test failures |
| 151 | + |
| 152 | +### Python Tests |
| 153 | +- Test files in `python/tests/` |
| 154 | +- Use pytest framework |
| 155 | +- Ensure Python bindings are built before running tests |
| 156 | + |
| 157 | +## Important Implementation Details |
| 158 | + |
| 159 | +### Tree Operations |
| 160 | +- The tree uses probabilistic balancing based on content hashes |
| 161 | +- Node splitting is determined by hash thresholds, not fixed size |
| 162 | +- All operations maintain Merkle tree properties for verification |
| 163 | + |
| 164 | +### Storage Abstraction |
| 165 | +- `NodeStorage` trait allows pluggable storage backends |
| 166 | +- Each backend implements get/put operations for nodes |
| 167 | +- Git backend stores nodes as Git objects for version control |
| 168 | + |
| 169 | +### Memory Management |
| 170 | +- Tree uses reference counting for node sharing |
| 171 | +- LRU cache available for frequently accessed nodes |
| 172 | +- Python bindings handle memory safely through PyO3 |
| 173 | + |
| 174 | +### Concurrency |
| 175 | +- Thread-safe variants available for multi-threaded access |
| 176 | +- Agent memory system uses Tokio for async operations |
| 177 | +- Git operations use file locking for concurrent access |
| 178 | + |
| 179 | +## Common Pitfalls & Solutions |
| 180 | + |
| 181 | +### Building Issues |
| 182 | +- Ensure Rust toolchain is installed: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` |
| 183 | +- For Python bindings, install maturin: `pip install maturin` |
| 184 | +- RocksDB feature requires system libraries on some platforms |
| 185 | + |
| 186 | +### Testing |
| 187 | +- Some tests require Git to be configured: `git config user.name "Test"` and `git config user.email "test@example.com"` |
| 188 | +- SQL tests may create temporary databases in `/tmp` |
| 189 | +- Agent tests may require OPENAI_API_KEY environment variable (can be dummy value for tests) |
| 190 | + |
| 191 | +### Performance |
| 192 | +- Use batch operations when inserting multiple keys |
| 193 | +- Enable LRU cache for read-heavy workloads |
| 194 | +- Consider RocksDB backend for large datasets |
| 195 | + |
| 196 | +## Project Dependencies |
| 197 | + |
| 198 | +### Critical Dependencies |
| 199 | +- `sha2`: Cryptographic hashing for Merkle tree |
| 200 | +- `serde` & `bincode`: Serialization for node storage |
| 201 | +- `gix`: Git integration (optional feature) |
| 202 | +- `gluesql-core`: SQL query engine (optional feature) |
| 203 | +- `pyo3`: Python bindings (optional feature) |
| 204 | +- `rocksdb`: Persistent storage backend (optional feature) |
0 commit comments