Merge pull request #12 from 0xmariowu/feat/v2-skills-architecture

feat: v2 skills-native self-evolving search architecture

Author: 0xmariowu

.claude/commands/autosearch.md

@@ -0,0 +1,28 @@
+# /autosearch — Self-Evolving Search
+
+Run a complete AutoSearch v2 session. Read and follow `autosearch/v2/PROTOCOL.md` as your operating system.
+
+## Arguments
+
+$ARGUMENTS — the search task (e.g., "find AI agent frameworks", "research vector databases", "discover new MCP servers")
+
+If no arguments provided, ask the user what to search for.
+
+## Execution
+
+1. Read `autosearch/v2/PROTOCOL.md` in full
+2. Read `autosearch/v2/state/config.json` for current parameters
+3. Follow the protocol exactly:
+   - Create task_spec from the user's input
+   - Run the AVO loop: PLAN → SEARCH → SCORE → DIAGNOSE → EVOLVE → RECORD
+   - Use Python 3.11+ to run `judge.py` (system python3 may be 3.9)
+   - Deliver when score meets threshold or generations exhausted
+4. Report results with evidence
+
+## Key constraints
+
+- `judge.py` is the only evaluator — never self-assess quality
+- All search APIs are free (gh, curl, ddgs)
+- State files are append-only (worklog.jsonl, patterns.jsonl)
+- Config/skill changes go through git commit/revert
+- Read the relevant skill file before executing any capability

CHANGELOG.md

@@ -5,6 +5,9 @@ All changes to AutoSearch. Every entry includes **what** changed and **why**.
 ---
 
 ## 2026-03-28
+- You can now run `/autosearch "find AI agent frameworks"` to execute a self-evolving search session. The system autonomously searches 5 platforms, scores results with judge.py, reflects, evolves its strategy, and delivers. Why: v2 replaces ~7000 lines of Python with ~100 lines of code + ~2500 lines of Markdown skills. Everything is now skills-native and AVO-driven.
+- Added `autosearch/v2/` — complete skills-native architecture: PROTOCOL.md (operating protocol), skill-spec.md (skill format), judge.py (deterministic scorer), 14 skills (5 platform + 4 strategy + 5 AVO), state management. Why: v1's Python runtime was hard to evolve; v2 lets the AVO loop modify skills directly as Markdown.
+- End-to-end validated: 43 results from 4 platforms, judge score 0.853, full worklog cycle from task_spec to delivery.
 - You can now evolve search strategies as JSON genomes instead of hardcoded Python. New `genome/` module: schema (8 sections), runtime interpreter, 13 primitives, 5 mutation operators, safe expression evaluator. Run `python3 avo.py "task" --genome` to start genome-based AVO evolution. Why: AVO paper's Vary(P_t) needs a structured, evolvable representation — Python code can't be safely mutated by an AI agent, but JSON genomes can.
 - Three seed genomes created from existing strategies: `engine-3phase.json` (3-phase explore/harvest/postmortem), `orchestrator-react.json` (ReAct loop), `daily-discovery.json` (daily discovery). Why: AVO needs initial population members to start evolution from known-good strategies.
 - All existing modules now accept optional `genome=` parameter with fallback to current hardcoded defaults. Files: modes.py, lexical.py, planner.py, synthesizer.py, project_experience.py, engine.py, orchestrator.py, daily.py, goal_runtime.py. Why: gradual migration — genome config is opt-in, existing behavior unchanged.

CLAUDE.md

@@ -72,6 +72,21 @@ AIMD
 8. Update `AIMD/experience/INDEX.jsonl` (path relative to Dev/ root).
 9. If anything changed beyond search runs (code, rules, architecture): prepend entry to `CHANGELOG.md`.
 
+## v2 rules (skills-native architecture)
+
+12. Don't modify `judge.py`. It is the fixed evaluation contract. AI must not self-assess quality — always run judge.py. Because: AVO paper §3.1 requires f to be a fixed contract; self-evaluation defeats the evolutionary signal.
+
+13. Don't modify AVO skill files (`skills/avo/*.md`), `PROTOCOL.md`, or `skill-spec.md`. These are immutable protocol definitions. Because: if the evolution engine can change its own rules, behavior becomes unpredictable.
+
+14. Don't delete or rewrite lines in `state/worklog.jsonl` or `state/patterns.jsonl`. They are append-only. Because: the AVO loop learns from history — deleting entries resets accumulated intelligence.
+
+15. Don't use paid APIs in platform skills. All search must use free access methods (gh CLI, curl to public APIs, ddgs). Because: zero marginal cost is a core design constraint.
+
+16. Config and skill changes during AVO must go through `git commit`. Failed changes get `git revert`. Because: git history IS the lineage that AVO uses to learn from failures.
+
+17. Use Python 3.11+ to run `judge.py` and tests. System python3 may be 3.9 which lacks union type syntax. Command: `~/.local/share/uv/python/cpython-3.11-macos-aarch64-none/bin/python3.11` or `uv run --python 3.11`.
+
 ---
 
 For methodology details, algorithm documentation, and system architecture, see `docs/` or inline comments in `engine.py`.
+For v2 architecture, see `autosearch/v2/PROTOCOL.md` (the system) and `autosearch/v2/skill-spec.md` (the format).

HANDOFF.md

@@ -1,53 +1,32 @@
 # AutoSearch Handoff
 
-## main
-
-**Last session**: 2026-03-27 (evening) — Capabilities System + AVO Evolution + MCP Server [SESSION-ID: 1526-reykjavik-quiet]
-
-**What was done**: Built AI-native capabilities registry (43 caps), orchestrator (OpenRouter ReAct loop), AVO evolution framework (arXiv:2603.24517), MCP server (6 tools). Fixed 20 bugs (6 infra + 14 audit). Researched 18 open-source search projects. AVO evolved search score 0.50→0.81, cumulative 321 URLs across 15 generations.
+## feat/v2-skills-architecture
 
-**Current state**: 194 tests pass. 43/43 capability self-tests pass. 12 commits on main. Evolved prompt at `sources/evolved-prompt.txt` (auto-loaded). MCP registered at `~/.mcp.json`.
+**Last session**: 2026-03-28 — v2 Skills Architecture: F001-F008 complete, PR ready
 
-**Key files**: `capabilities/` (43 files), `orchestrator.py`, `avo.py`, `autosearch-mcp/` (3 files), `docs/2026-03-27-deep-research-patterns.md`
+### Completed
 
-**Usage**: `python3 cli.py --orchestrated "find repos" --max-steps 15` | `python3 avo.py "find 200 repos" --generations 10` | MCP: restart Claude Code, say "搜一下 XX"
+- F001: PROTOCOL.md + skill-spec.md + config.json + directory structure
+- F002: 5 platform skills (github, web-ddgs, reddit, hackernews, arxiv)
+- F003: 4 strategy skills (query-expand, score, deduplicate, synthesize)
+- F004: judge.py (157 lines, 12 tests passing) — the only code
+- F005: 5 AVO self-evolution skills (reflect, evolve, diagnose, create-skill, stuck)
+- F006: End-to-end validated — 43 results, score 0.853, full worklog cycle
+- F007: `/autosearch` Claude Code skill entry point
+- F008: CLAUDE.md v2 rules, CHANGELOG, plan moved to completed
 
-**Next**: MCP e2e verify (restart session), AVO 20+ gen, add OPENROUTER_API_KEY to shell profile, fix remaining 400 errors
+### Known Issues
 
-## worktree-1037-zanzibar-teal
+- ddgs (DuckDuckGo) fails on Python 3.9 due to SSL — works on 3.11+
+- judge.py requires Python 3.11+ (union type syntax)
+- arXiv skill fixed to use HTTPS
 
-**Last session**: 2026-03-27. Two workstreams completed.
+### Next Steps
 
-### Workstream 1: AI Judge Pipeline Fixes (DONE)
-- PR #4 merged: 5 bugs fixed — `_bundle_sample()` rich content, content truncation 500→3000, `_dimension_aware_bundle_sample()`, `search_for_gaps()` acquisition policy, `scripts/batch_enrich.py`
-- PR #5 merged: content limit 1500→3000
-- Batch enriched 60 evidence records (3% → 35% with page content)
-- Ran evolve 5+3 rounds: score stuck at 68. Root cause = evidence relevance, not pipeline
-- Cleaned 4 stale tests, fixed pre-push hook (pytest → unittest)
+- Merge PR to main
+- F007-S3: MCP server (thin shell, optional)
+- F007-S4: Cron/schedule entry (optional)
 
-### Workstream 2: BaZi Case Collection (IN PROGRESS)
-- **Goal**: Collect 200K real BaZi (八字) fortune-telling case studies from the internet
-- **Infrastructure**: Set up SearXNG Docker (port 8888) + installed ddgs. All 12 AutoSearch backends now online.
-- **Goal case**: `goal_cases/bazi-case-collection.json` — 4 dimensions (birth_data, four_pillars, analysis_content, case_depth)
-- **Evidence**: `goal_cases/runtime/bazi-case-collection/evidence-index.jsonl` — **6,757 evidence pages** (39MB)
-  - 1,132 with page content (fit_markdown/acquired_text)
-  - 1,098 unique domains
-  - Top sources: 知乎专栏 (587), 抖音 (355), 搜狐 (336), 百家号 (215), B站 (199), 360doc (125)
-- **Queries run**: ~3,718 unique queries across 8 parallel batches, zero errors
-- **Search saturation**: Dedup rate very high at end — public Chinese web BaZi URLs approaching coverage ceiling
-
-**Next step for BaZi collection**:
-1. **Content extraction**: 6,757 pages collected but only 1,132 have page content. Run `scripts/batch_enrich.py --goal-case bazi-case-collection --limit 3000` to fetch page content for top records
-2. **Case parsing**: Each page may contain multiple individual cases. Need extraction script to pull structured BaZi data (birth datetime, four pillars, analysis text) from page content
-3. **More queries**: Generated 3,315 queries but could expand to 10K+ by adding more combinatorial dimensions (specific 六十甲子 year combos, more site targets, pagination patterns)
-4. **Quality filtering**: Filter evidence by dimension keyword coverage to keep only pages that actually contain case data
-
-**Key files**:
-- `goal_cases/bazi-case-collection.json` — goal case definition
-- `goal_cases/runtime/bazi-case-collection/evidence-index.jsonl` — collected evidence (39MB, 6,757 records)
-- `chinese-divination-knowledge-bases.md` — comprehensive catalog of ~65 open-source divination projects (also copied to repo root)
-- `/tmp/bazi_queries_10k.json` — 3,315 generated queries (NOTE: in /tmp, may not survive reboot — regenerate with `/tmp/gen_bazi_queries.py`)
-- `/tmp/bazi_batch_runner.py` — parallel batch search runner
-- `/tmp/gen_bazi_queries.py` — combinatorial query generator
+## main
 
-**SearXNG**: Docker container `searxng` on port 8888. Config at `/tmp/searxng/settings.yml` (JSON API enabled, limiter off). Container may need restart after reboot: `docker start searxng`
+v1 Genome architecture live (PR #11 + PR #9). 194 tests passing.