v0.2.0 - Iterator API & Performance

🚀 New Features

Iterator API

Added memory-efficient iterator API for processing large documents:

# Block syntax - most efficient (no array allocation)
splitter.each_chunk(text) { |chunk| process(chunk) }

# Iterator with lazy evaluation
splitter.each_chunk(text).first(10).each { |chunk| ... }

# Traditional array (backward compatible)
chunks = splitter.split_text(text)

⚡ Performance Improvements

• 25% faster: Process 1MB in 6.79ms (was 9ms)
• 57% less memory: Only 17.9MB/op (was 41.9MB/op)
• Uses String::Builder for efficient string construction
• Optimized with Crystal's native split method

Benchmark Results (1MB text)

• Throughput: 147 ops/sec
• Latency: 6.79ms
• Memory: 17.9MB per operation
• Chunks: 1,249

📚 Documentation

• Added iterator API examples and usage patterns
• Updated README with performance benchmarks
• Added examples/iterator_usage.cr with practical examples
• Updated feature comparison table

🔄 Backward Compatibility

All existing code continues to work. The split_text() method returns an Array as before.

💡 Benefits

1. Memory efficiency: Process huge documents without loading all chunks into memory
2. Streaming capable: Lazy evaluation with iterator chains
3. Production ready: All 25 tests passing
4. Type safe: Full Crystal type safety

---

Full Changelog: v0.1.1...v0.2.0