Add MCP server with pagination and TUI refactoring

This commit adds comprehensive MCP (Model Context Protocol) support and refactors the TUI into a dedicated crate.

## MCP Server Features
- Full stdio transport implementation with tool discovery
- Pagination support with cursor-based navigation (1-200 results per page)
- Session management with TTL and automatic cleanup (60s expiration)
- Search tools: semantic_search, regex_search, hybrid_search, lexical_search
- Index tools: index_status, reindex, health_check
- Fallback strategies: automatic semantic → lexical when embeddings unavailable
- 7 comprehensive integration tests for pagination and edge cases

## TUI Refactoring
- Extracted 3,084 lines of TUI code into dedicated ck-tui crate
- Clean modular architecture with public API
- Config persistence to ~/.config/ck/tui.json
- Multiple preview modes (Heatmap, Syntax, Chunks)
- Multiple search modes (Semantic, Regex, Hybrid)
- Multi-select workflow and editor integration
- Real-time indexing progress with file/chunk counts

## CLI Enhancements
- Color-coded similarity scores with RGB gradient heatmap
- Enhanced visual output with Unicode box drawing
- Near-miss tracking for threshold queries

## Core Improvements
- Span validation with Span::new() to prevent invalid spans
- Mixed line ending support (Unix \n, Windows \r\n, Mac \r)
- Memory-efficient streaming file operations
- Better PDF content path resolution

## Technical Details
- Session cleanup: LRU-based eviction with periodic task (30s)
- Line ending tracking: split_lines_with_endings() for byte precision
- SearchResults enhancement: closest_below_threshold field
- Full test coverage with integration tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Author: runonthespot

CHANGELOG.md

@@ -2,6 +2,40 @@
 
 All notable changes to this project will be documented in this file.
 
+## [0.6.0] - 2025-10-12
+
+### Added
+- **MCP Server**: Full Model Context Protocol implementation with stdio transport for AI agent integration
+- **Pagination Support**: Cursor-based pagination for all search modes (page_size: 1-200, default: 50)
+- **Session Management**: TTL-based session cleanup for paginated results with automatic expiration (60s)
+- **MCP Search Tools**: `semantic_search`, `regex_search`, `hybrid_search`, `lexical_search` with unified interface
+- **MCP Index Tools**: `index_status`, `reindex`, `health_check` for index management
+- **CLI Heatmap Visualization**: Color-coded similarity scores with RGB gradient highlighting (red→yellow→green)
+- **Enhanced Visual Output**: Unicode box drawing and sophisticated match highlighting in CLI
+- **Near-Miss Tracking**: Track closest result below threshold for better search feedback
+- **Fallback Strategies**: Automatic fallback from semantic to lexical search when embeddings unavailable
+- **Graceful Error Handling**: Skip stale index entries when files no longer exist
+
+### Fixed
+- **Mixed Line Endings**: Proper handling of Unix (\n), Windows (\r\n), and Mac (\r) line endings
+- **Span Validation**: Prevent invalid spans with zero line numbers using `Span::new()` validation
+- **Streaming File Operations**: Memory-efficient line extraction without loading entire files
+- **PDF Content Resolution**: Better content path resolution and caching for PDF files
+
+### Technical
+- **MCP Protocol**: Full implementation with tool discovery, validation, and error handling
+- **Session Cleanup**: LRU-based eviction with periodic cleanup task (every 30s)
+- **Streaming Reads**: Optimized `extract_lines_from_file()` for minimal memory footprint
+- **SearchResults Enhancement**: Added `closest_below_threshold` field for improved UX
+- **Comprehensive Testing**: 7 new MCP integration tests covering pagination, validation, and edge cases
+- **Line Ending Support**: `split_lines_with_endings()` tracks exact byte lengths per line
+- **TUI Refactoring**: Extracted TUI functionality into dedicated `ck-tui` crate (3,084 lines)
+- **Modular Architecture**: Clean separation of TUI components with public API
+- **Config Persistence**: TUI preferences saved to `~/.config/ck/tui.json`
+
+### Breaking Changes
+- **Span Construction**: Use `Span::new()` for validated construction instead of struct literals (backward compatible via `Span::new_unchecked()`)
+
 ## [0.5.3] - 2025-09-29
 
 ### Added

Cargo.lock

@@ -9,10 +9,11 @@ members = [
     "ck-embed",
     "ck-ann",
     "ck-models",
+    "ck-tui",
 ]
 
 [workspace.package]
-version = "0.5.3"
+version = "0.6.0"
 edition = "2024"
 authors = ["Mike Renwick"]
 license = "MIT OR Apache-2.0"

Cargo.toml

@@ -67,6 +67,29 @@ claude mcp list  # or use /mcp in Claude Code
 
 **Built-in Pagination:** Handles large result sets gracefully with page_size controls, cursors, and snippet length management.
 
+### 🎨 **Interactive TUI (Terminal User Interface)**
+Launch an interactive search interface with real-time results and multiple preview modes:
+
+```bash
+# Start TUI for current directory
+ck --tui
+
+# Start with initial query
+ck --tui "error handling"
+```
+
+**Features:**
+- **Multiple Search Modes**: Toggle between Semantic, Regex, and Hybrid search with `Tab`
+- **Preview Modes**: Switch between Heatmap, Syntax highlighting, and Chunk view with `Ctrl+V`
+- **View Options**: Toggle between snippet and full-file view with `Ctrl+F`
+- **Multi-select**: Select multiple files with `Ctrl+Space`, open all in editor with `Enter`
+- **Search History**: Navigate with `Ctrl+Up/Down`
+- **Editor Integration**: Opens files in `$EDITOR` with line numbers (Vim, VS Code, Cursor, etc.)
+- **Progress Tracking**: Live indexing progress with file and chunk counts
+- **Config Persistence**: Preferences saved to `~/.config/ck/tui.json`
+
+See [TUI.md](TUI.md) for keyboard shortcuts and detailed usage.
+
 ### 🔍 **Semantic Search**
 Find code by concept, not keywords. Understands synonyms, related terms, and conceptual similarity:
 
@@ -348,6 +371,7 @@ ck --json --sem "public API" src/ | generate_docs.py
 ck uses a modular Rust workspace:
 
 - **`ck-cli`** - Command-line interface and MCP server
+- **`ck-tui`** - Interactive terminal user interface (ratatui-based)
 - **`ck-core`** - Shared types, configuration, and utilities
 - **`ck-engine`** - Search engine implementations (regex, semantic, hybrid)
 - **`ck-index`** - File indexing, hashing, and sidecar management