feat: production-ready polish

AGENTS.md

@@ -1,102 +1,64 @@
-# SemanticCache - AI Agent Instructions
+# SemanticCache -- Agent Instructions
 
-This is a high-performance semantic caching library for Go that uses vector embeddings to find semantically similar content.
+Semantic caching library for Go. Sync-only `Cache[K, V]` with pluggable backends and embedding providers.
 
-## Dev environment tips
-- Use `go mod tidy` to ensure dependencies are up to date
-- Run `go test ./...` to execute the full test suite across all packages
-- Use `go test -cover ./...` for test coverage analysis
-- Run `go build ./...` to verify all packages compile correctly
-- Use `go run` with the main examples for quick testing
-
-## Project structure
-The project follows Go best practices with modular package organization:
-- `cache.go` - Main cache implementation with `New()` function
-- `options/` - All functional options (`options.With*()` functions)
-- `similarity/` - Similarity algorithms (`similarity.*Similarity` functions)  
-- `backends/` - Storage backends (in-memory and Redis)
-- `providers/` - Embedding providers (OpenAI)
-- `types/` - Shared interfaces and types
-
-## Testing instructions
-- Run the full test suite: `go test ./...`
-- Test specific packages: `go test ./cache_test.go ./cache.go -v`
-- Run benchmarks: `go test -bench=. ./...`
-- Check test coverage: `go test -cover ./...`
-- All tests must pass before merging changes
-- Add tests for any new functionality you implement
-- Use mock providers for testing without external API dependencies
-
-## Code style guidelines
-- Follow standard Go conventions and idioms
-- Use the functional options pattern for configuration
-- Maintain generic type support throughout the API
-- All public functions and types must have Go doc comments
-- Use context.Context for all operations that might be cancelled
-- Keep interface segregation principle - small, focused interfaces
-
-## Build and development commands
-- `go mod download` - Download dependencies
-- `go build ./...` - Build all packages
-- `go test ./...` - Run all tests
-- `go test -race ./...` - Run tests with race detection
-- `go vet ./...` - Run static analysis
-- `go fmt ./...` - Format code
+## Commands
+```
+go test ./...             # all tests
+go test -race ./...       # race detector
+go test -bench=. ./...    # benchmarks
+go test -cover ./...      # coverage
+go vet ./...              # static analysis
+gofmt -l .                # formatting check
+go build ./...            # build all
+```
 
-## Package import structure
-When adding new code, use these imports:
+## Import path
 ```go
-import (
-    "github.com/botirkhaltaev/semanticcache"           // Main cache
-    "github.com/botirkhaltaev/semanticcache/options"   // Configuration options
-    "github.com/botirkhaltaev/semanticcache/similarity" // Similarity algorithms
-    "github.com/botirkhaltaev/semanticcache/types"     // Shared types
-)
+import "github.com/botirk38/semanticcache"
 ```
 
-## Performance considerations
-- LRU backend for time-locality patterns
-- LFU backend for frequency-based access patterns  
-- FIFO backend for simple queue-like usage
-- Use batch operations (`SetBatch`, `GetBatch`) for multiple items
-- Adjust similarity thresholds based on precision requirements
-- Always use context for request timeouts and cancellation
-
-## Common development tasks
+Subpackages: `options`, `types`, `backends/inmemory`, `backends/remote`, `providers/openai`, `providers/local`, `similarity`, `chunker`, `tokenizer`.
 
-### Adding new similarity algorithms
-1. Create new file in `similarity/` package
-2. Implement function with signature: `func(a, b []float32) float32`
-3. Add comprehensive tests in `similarity_test.go`
-4. Export function for use in options
+## Architecture
+- `cache.go` + `errors.go` -- `Cache[K, V]` type, constructors, sentinel errors (`ErrClosed`, `ErrZeroKey`, `ErrInvalidN`)
+- `types/` -- `Backend[K, V]` interface (9 methods), `EmbeddingProvider`, `BatchEmbeddingProvider`
+- `options/` -- functional options (`With*` functions), config errors (`ErrNilBackend`, `ErrNilProvider`, `ErrNilComparator`)
+- `backends/inmemory/` -- LRU, LFU, FIFO (thread-safe via `sync.RWMutex`)
+- `backends/remote/` -- Redis (JSON storage, requires RedisJSON or Redis 7.2+)
+- `providers/openai/` -- OpenAI SDK, default model `text-embedding-3-small`
+- `providers/local/` -- hash-based provider for testing (no API key, not semantically meaningful)
+- `similarity/` -- `func(a, b []float64) float64` functions (cosine, euclidean, dot, manhattan, pearson)
+- `chunker/` -- text chunking with configurable strategy, its own errors
+- `tokenizer/` -- token counting for OpenAI (local), Anthropic (API), Gemini (API)
 
-### Adding new backends
-1. Implement the `Backend` interface from `types/types.go`
-2. Create corresponding option function in `options/options.go`
-3. Add integration tests with the main cache
-4. Update documentation and examples
+## Error conventions
+Each package defines its own errors. No centralized errors package.
 
-### Adding new providers
-1. Implement the `EmbeddingProvider` interface from `types/types.go`
-2. Create option function in `options/options.go`
-3. Add tests with mock implementations
-4. Consider rate limiting and error handling
+## Adding a backend
+1. Implement all 9 methods of `types.Backend[K, V]`
+2. Add `With*Backend` option in `options/options.go`
+3. Add compile-time check: `var _ types.Backend[string, string] = (*YourBackend[string, string])(nil)`
+4. Add tests + benchmarks
+5. Add re-export in `backends/backends.go`
+6. Add `README.md` and `AGENTS.md` in the new package
 
-## Error handling
-- Return descriptive errors for configuration issues
-- Handle network timeouts and API rate limits gracefully
-- Validate inputs and provide clear error messages
-- Use Go's standard error handling patterns
+## Adding a provider
+1. Implement `types.EmbeddingProvider` (`EmbedText`, `Close`)
+2. Optionally implement `types.BatchEmbeddingProvider`
+3. Add `With*Provider` option in `options/options.go`
+4. Add tests (use `httptest` for HTTP-based providers)
+5. Add re-export in `providers/providers.go`
+6. Add `README.md` and `AGENTS.md` in the new package
 
-## Security considerations
-- Never log or expose API keys or sensitive data
-- Validate all inputs to prevent injection attacks
-- Use secure defaults for configurations
-- Handle authentication failures appropriately
+## Adding a similarity function
+1. New file in `similarity/`, signature `func(a, b []float64) float64`
+2. Return 0 for mismatched lengths or empty vectors
+3. Add tests in `similarity_test.go`
 
-## Dependencies
-The project uses minimal external dependencies:
-- Standard library for core functionality
-- Context package for cancellation and timeouts
-- Generic type support (Go 1.18+)
-- External APIs (OpenAI) are optional and pluggable
+## Code style
+- Standard Go conventions. `gofmt` enforced in CI.
+- Functional options pattern for configuration.
+- `context.Context` first argument on all I/O methods.
+- Generics: `[K comparable, V any]` throughout.
+- Each module has its own `README.md` (for humans) and `AGENTS.md` (for agents).

CLAUDE.md

@@ -1,108 +1,71 @@
-# SemanticCache - Claude Development Instructions
+# SemanticCache -- Development Instructions
 
-## Project Overview
-This is a high-performance semantic caching library for Go that uses vector embeddings to find semantically similar content. The library supports multiple backends (in-memory and Redis) and embedding providers (OpenAI).
-
-## Project Structure
-The project is organized into modular packages following Go best practices:
+## Overview
+Semantic caching library for Go. Stores values with embedding vectors, retrieves by similarity. Sync-only `Cache[K, V]` type with pluggable backends and embedding providers.
 
+## Project structure
 ```
 semanticcache/
-├── similarity/                    # Similarity algorithms package
-│   ├── similarity.go             # SimilarityFunc type definition
-│   ├── cosine.go                 # Individual algorithm implementations
-│   ├── euclidean.go, dotproduct.go, manhattan.go, pearson.go
-│   └── similarity_test.go        # Complete test suite
-├── options/                       # Functional options package  
-│   ├── options.go                # All With* option functions
-│   └── options_test.go           # Options test suite
-├── backends/                      # Storage backends
-│   ├── inmemory/                 # In-memory backends (LRU, LFU, FIFO)
-│   └── remote/                   # Remote backends (Redis)
-├── providers/                     # Embedding providers
-│   └── openai/                   # OpenAI provider implementation
-├── types/                        # Shared types and interfaces
-├── cache.go                      # Main cache implementation
-├── cache_test.go                 # Main cache tests
-└── README.md                     # Updated with new structure
+  cache.go, errors.go          Cache type, constructors, sentinel errors
+  types/                       Backend[K,V] and EmbeddingProvider interfaces
+  options/                     Functional options (With* functions), config errors
+  backends/
+    inmemory/                  LRU, LFU, FIFO (thread-safe)
+    remote/                    Redis (JSON storage)
+  providers/
+    openai/                    OpenAI embeddings (official SDK)
+    local/                     Hash-based provider for testing (no API key)
+  similarity/                  Cosine, Euclidean, DotProduct, Manhattan, Pearson
+  chunker/                     Text chunking utilities
+  tokenizer/                   Token counting (OpenAI, Anthropic, Gemini)
 ```
 
-## Development Guidelines
-
-### Package Usage
-- **Main cache**: `semanticcache.New()` - creates new cache instances
-- **Options**: `options.With*()` - all functional options for configuration
-- **Similarity**: `similarity.*Similarity` - similarity algorithm functions
-- **Types**: `types.*` - shared interfaces and types
-
-### Testing Commands
-- Run all tests: `go test ./...`
-- Run specific package tests: `go test ./cache_test.go ./cache.go -v`
-- Test coverage: `go test -cover ./...`
-
-### Code Style
-- Follow Go idioms and conventions
-- Use functional options pattern for configuration
-- Maintain backward compatibility when possible
-- Add comprehensive tests for new features
-- Document public APIs with Go comments
-
-### Key Design Patterns
-1. **Functional Options**: All configuration uses the options pattern
-2. **Interface Segregation**: Separate interfaces for backends, providers, similarity functions
-3. **Generic Types**: Full generic support for key/value types
-4. **Context Awareness**: All operations support context.Context
-5. **Modular Architecture**: Clear separation of concerns across packages
-
-### Common Tasks
-
-#### Adding New Similarity Algorithm
-1. Create new file in `similarity/` package
-2. Implement function with signature `func(a, b []float32) float32`
-3. Add tests in `similarity_test.go`
-4. Export function for use in options
+## Key design decisions
+- Sync-only. No async cache.
+- `Backend[K, V]` has 9 methods. Every backend implements all of them.
+- No centralized errors package. Each package defines its own errors.
+- Functional options pattern for configuration.
+- `context.Context` on all operations.
+- Generics throughout: `Cache[K comparable, V any]`.
 
-#### Adding New Backend
-1. Implement the `Backend` interface in `types/types.go`
-2. Create option function in `options/options.go`
-3. Add comprehensive tests
-4. Update documentation
-
-#### Adding New Provider
-1. Implement the `EmbeddingProvider` interface in `types/types.go`
-2. Create option function in `options/options.go`
-3. Add integration tests
-4. Update documentation
-
-### Testing Strategy
-- Unit tests for individual components
-- Integration tests for end-to-end functionality
-- Benchmark tests for performance-critical paths
-- Mock providers for testing without external dependencies
+## Commands
+```
+go test ./...             # all tests
+go test -race ./...       # with race detector
+go test -bench=. ./...    # benchmarks
+go vet ./...              # static analysis
+gofmt -l .                # formatting check
+go build ./...            # build all
+```
 
-### Import Guidelines
-When working with the codebase:
+## Import path
 ```go
-import (
-    "github.com/botirkhaltaev/semanticcache"
-    "github.com/botirkhaltaev/semanticcache/options"
-    "github.com/botirkhaltaev/semanticcache/similarity"
-    "github.com/botirkhaltaev/semanticcache/types"
-)
+import "github.com/botirk38/semanticcache"
 ```
 
-### Recent Restructuring
-The project was recently restructured from a monolithic approach to modular packages:
-- Moved similarity functions to `similarity/` package
-- Moved functional options to `options/` package
-- Updated all imports and references throughout the codebase
-- Maintained API compatibility where possible
-
-### Build and Test
-Ensure all tests pass before making changes:
-```bash
-go test ./...
-go build ./...
-```
+All subpackages: `github.com/botirk38/semanticcache/{options,types,backends/inmemory,...}`
+
+## Adding a new backend
+1. Implement `types.Backend[K, V]` (9 methods)
+2. Add `With*Backend` option in `options/options.go`
+3. Add compile-time check: `var _ types.Backend[string, string] = (*YourBackend[string, string])(nil)`
+4. Add tests in the backend's package
+5. Add a re-export in `backends/backends.go`
+
+## Adding a new provider
+1. Implement `types.EmbeddingProvider` (2 methods: `EmbedText`, `Close`)
+2. Optionally implement `types.BatchEmbeddingProvider`
+3. Add `With*Provider` option in `options/options.go`
+4. Add tests (use `httptest` for HTTP providers)
+5. Add a re-export in `providers/providers.go`
+
+## Adding a similarity function
+1. Create a new file in `similarity/`
+2. Implement `func(a, b []float64) float64`
+3. Add tests in `similarity_test.go`
 
-The project should build without warnings and all tests should pass.
+## Error conventions
+- Each package defines its own sentinel errors.
+- Root package: `ErrClosed`, `ErrZeroKey`, `ErrInvalidN`
+- Options: `ErrNilBackend`, `ErrNilProvider`, `ErrNilComparator`
+- Chunker: `ErrInvalidChunkSize`, `ErrEmptyText`, etc.

CONTRIBUTING.md

@@ -67,7 +67,7 @@ gofmt -l .
 
 ## Adding a New Backend
 
-1. Implement the `types.CacheBackend[K, V]` interface.
+1. Implement the `types.Backend[K, V]` interface.
 2. Add a constructor in `backends/`.
 3. Add an `options.With*Backend` function in `options/options.go`.
 4. Add integration tests.

Makefile

@@ -0,0 +1,29 @@
+.PHONY: test test-race bench vet fmt lint build clean
+
+test:
+	go test ./...
+
+test-race:
+	go test -race ./...
+
+bench:
+	go test -bench=. -benchmem ./...
+
+vet:
+	go vet ./...
+
+fmt:
+	gofmt -l .
+
+fmt-fix:
+	gofmt -w .
+
+lint: vet fmt
+
+build:
+	go build ./...
+
+clean:
+	go clean -testcache
+
+check: build test-race vet fmt