feat: add structured output and history processors

CLAUDE.md

@@ -43,11 +43,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - `SubAgentToolset`: Spawn and delegate to subagents
 - `SkillsToolset`: Load and use skill definitions from markdown files
 
+**Processors (`pydantic_deep/processors/`)**
+- `SummarizationProcessor`: Automatic conversation summarization for token management
+- `create_summarization_processor()`: Factory function for creating summarization processors
+
 **Types (`pydantic_deep/types.py`)**
 - Pydantic models for all data structures
 - `FileData`, `FileInfo`, `WriteResult`, `EditResult`, `GrepMatch`
 - `Todo`, `SubAgentConfig`, `CompiledSubAgent`
 - `Skill`, `SkillDirectory`, `SkillFrontmatter`
+- `ResponseFormat`: Alias for structured output specification
 
 ### Key Design Patterns
 
@@ -86,6 +91,33 @@ deps = DeepAgentDeps(
 )
 ```
 
+**Structured Output**
+```python
+from pydantic import BaseModel
+from pydantic_deep import create_deep_agent
+
+class TaskResult(BaseModel):
+    status: str
+    details: str
+
+# Agent returns TaskResult instead of str
+agent = create_deep_agent(output_type=TaskResult)
+```
+
+**Context Management / Summarization**
+```python
+from pydantic_deep import create_deep_agent
+from pydantic_deep.processors import create_summarization_processor
+
+# Automatically summarize when reaching token limits
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),  # or ("messages", 50) or ("fraction", 0.8)
+    keep=("messages", 20),       # Keep last N messages after summarization
+)
+
+agent = create_deep_agent(history_processors=[processor])
+```
+
 ## Testing Strategy
 
 - **Unit tests**: `tests/` directory with comprehensive coverage

README.md

@@ -13,6 +13,8 @@ Deep agent framework built on [pydantic-ai](https://github.com/pydantic/pydantic
 - **Multiple Backends**: StateBackend (in-memory), FilesystemBackend, DockerSandbox, CompositeBackend
 - **Rich Toolsets**: TodoToolset, FilesystemToolset, SubAgentToolset, SkillsToolset
 - **Skills System**: Extensible skill definitions with markdown prompts
+- **Structured Output**: Type-safe responses with Pydantic models via `output_type`
+- **Context Management**: Automatic conversation summarization for long sessions
 - **Human-in-the-Loop**: Built-in support for human confirmation workflows
 - **Streaming**: Full streaming support for agent responses
 
@@ -55,6 +57,42 @@ async def main():
 asyncio.run(main())
 ```
 
+## Structured Output
+
+Get type-safe responses with Pydantic models:
+
+```python
+from pydantic import BaseModel
+from pydantic_deep import create_deep_agent, create_default_deps
+
+class TaskAnalysis(BaseModel):
+    summary: str
+    priority: str
+    estimated_hours: float
+
+agent = create_deep_agent(output_type=TaskAnalysis)
+deps = create_default_deps()
+
+result = await agent.run("Analyze this task: implement user auth", deps=deps)
+print(result.output.priority)  # Type-safe access
+```
+
+## Context Management
+
+Automatically summarize long conversations to manage token limits:
+
+```python
+from pydantic_deep import create_deep_agent
+from pydantic_deep.processors import create_summarization_processor
+
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),  # Summarize when reaching 100k tokens
+    keep=("messages", 20),       # Keep last 20 messages
+)
+
+agent = create_deep_agent(history_processors=[processor])
+```
+
 ## Documentation
 
 - **[Full Documentation](https://vstorm-co.github.io/pydantic-deep/)** - Complete guides and API reference

docs/advanced/processors.md

@@ -0,0 +1,193 @@
+# History Processors
+
+pydantic-deep supports history processors for managing conversation context. The most common use case is automatic summarization to handle long conversations without exceeding token limits.
+
+## Summarization Processor
+
+The `SummarizationProcessor` monitors conversation length and automatically summarizes older messages when thresholds are reached.
+
+### Basic Usage
+
+```python
+from pydantic_deep import create_deep_agent
+from pydantic_deep.processors import create_summarization_processor
+
+# Create a summarization processor
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),  # Summarize when reaching 100k tokens
+    keep=("messages", 20),       # Keep last 20 messages after summarization
+)
+
+# Create agent with the processor
+agent = create_deep_agent(
+    history_processors=[processor],
+)
+```
+
+### Trigger Conditions
+
+You can trigger summarization based on different criteria:
+
+```python
+# Trigger when message count exceeds threshold
+processor = create_summarization_processor(
+    trigger=("messages", 50),
+)
+
+# Trigger when token count exceeds threshold
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),
+)
+
+# Trigger at fraction of max input tokens
+processor = create_summarization_processor(
+    trigger=("fraction", 0.8),  # 80% of max tokens
+    max_input_tokens=200000,    # Required for fraction triggers
+)
+
+# Multiple trigger conditions (any condition triggers)
+processor = create_summarization_processor(
+    trigger=[
+        ("messages", 100),
+        ("tokens", 150000),
+    ],
+)
+```
+
+### Retention Configuration
+
+Control how much context to keep after summarization:
+
+```python
+# Keep last N messages
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),
+    keep=("messages", 20),
+)
+
+# Keep last N tokens worth of messages
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),
+    keep=("tokens", 10000),
+)
+
+# Keep fraction of max tokens
+processor = create_summarization_processor(
+    trigger=("fraction", 0.8),
+    keep=("fraction", 0.1),
+    max_input_tokens=200000,
+)
+```
+
+### Custom Token Counter
+
+By default, the processor uses a simple character-based estimation (~4 characters per token). For more accurate counting, provide a custom token counter:
+
+```python
+def count_tokens(messages):
+    """Custom token counter using tiktoken or similar."""
+    import tiktoken
+    enc = tiktoken.get_encoding("cl100k_base")
+
+    total = 0
+    for msg in messages:
+        # Extract text from message and count tokens
+        # Implementation depends on your needs
+        pass
+    return total
+
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),
+    token_counter=count_tokens,
+)
+```
+
+### Custom Summary Prompt
+
+Customize how the summarization is performed:
+
+```python
+custom_prompt = """
+Extract the key information from this conversation.
+Focus on:
+- User requirements and goals
+- Important decisions made
+- Current state of the task
+
+Messages:
+{messages}
+
+Provide a concise summary.
+"""
+
+processor = create_summarization_processor(
+    trigger=("tokens", 100000),
+    summary_prompt=custom_prompt,
+)
+```
+
+## Using the Processor Class Directly
+
+For more control, use `SummarizationProcessor` directly:
+
+```python
+from pydantic_deep.processors import SummarizationProcessor
+
+processor = SummarizationProcessor(
+    model="anthropic:claude-sonnet-4-20250514",
+    trigger=("tokens", 100000),
+    keep=("messages", 20),
+    max_input_tokens=None,
+    trim_tokens_to_summarize=4000,  # Limit summary input size
+)
+```
+
+## How It Works
+
+1. **Before each model call**, the processor checks if any trigger condition is met
+2. If triggered, it finds a safe cutoff point that doesn't split tool call/response pairs
+3. Older messages are summarized using a lightweight LLM call
+4. The summary replaces the old messages, preserving recent context
+5. The agent continues with the compressed history
+
+### Tool Call Safety
+
+The processor ensures tool calls and their responses stay together:
+
+```
+Messages: [User, AI+ToolCall, ToolResponse, User, AI+ToolCall, ToolResponse, User]
+                                           ↑ Safe cutoff point (between complete pairs)
+```
+
+## Multiple Processors
+
+You can chain multiple history processors:
+
+```python
+from pydantic_deep import create_deep_agent
+from pydantic_deep.processors import create_summarization_processor
+
+# Multiple processors are applied in order
+agent = create_deep_agent(
+    history_processors=[
+        create_summarization_processor(trigger=("tokens", 100000)),
+        # Add more processors as needed
+    ],
+)
+```
+
+## Best Practices
+
+1. **Choose appropriate thresholds**: Set trigger thresholds below your model's context limit to leave room for the response
+
+2. **Keep enough context**: Retain sufficient recent messages for the agent to understand the current task
+
+3. **Monitor summarization quality**: Check that summaries preserve important context for your use case
+
+4. **Use fraction-based triggers for portability**: When switching between models with different context limits
+
+## Next Steps
+
+- [Structured Output](structured-output.md) - Type-safe responses with Pydantic models
+- [Streaming](streaming.md) - Real-time response handling
+- [Human-in-the-Loop](human-in-the-loop.md) - Approval workflows