Testing Strategy

Philosophy

Build components sequentially. Test each in isolation. Finish with integration tests.

Test Structure

internal/                              ← Unit tests (per package)
├── indexer/
│   ├── normalize.go
│   ├── normalize_test.go
│   ├── parse.go
│   ├── parse_test.go
│   ├── chunk.go
│   └── chunk_test.go
├── embedding/
│   ├── ollama.go
│   └── ollama_test.go         ← Mock HTTP, skip real Ollama
├── search/
│   ├── bm25.go
│   ├── bm25_test.go
│   ├── vector.go
│   └── vector_test.go
├── db/
│   ├── db.go
│   └── db_test.go             ← Use :memory: database
└── commands/
    └── index.go

integration/                           ← Integration tests (task-oriented)
├── index_documentation_from_files_test.go
├── get_documentation_from_github_test.go
└── search_documentation_test.go

Unit Test Guidelines

Isolation

Each component tests independently. No external dependencies in unit tests.

// Good: Test normalize in isolation
func TestNormalize_CRLF(test *testing.T) {
    input := []byte("line1\r\nline2\r\n")
    result := Normalize(input)
    if strings.Contains(result.Content, "\r") {
        test.Error("CRLF not converted")
    }
}

// Bad: Unit test that requires Ollama running
func TestChunk_WithEmbedding(test *testing.T) {
    // Don't do this - embedding is separate concern
}

Test Real Data

Use actual Svelte documentation snippets for realistic tests:

func TestParse_SvelteStateDoc(test *testing.T) {
    content := `## $state

Reactive state is declared with the $state rune:

` + "```svelte" + `
<script>
    let count = $state(0);
</script>
` + "```" + `
`
    doc, error := Parse("03-runes/01-state.md", content)
    // Assert structure...
}

Edge Cases

Test developer-specific tokens that standard parsers mishandle:

var DEV_TOKEN_CASES = []string{
    "$state",
    "$derived",
    "$effect.pre",
    "+page.server.ts",
    "+layout.svelte",
    "@sveltejs/kit",
    "on:click",
    "bind:value",
    "use:action",
}

func TestTokenizer_DevTokens(test *testing.T) {
    for _, token := range DEV_TOKEN_CASES {
        tokens := Tokenize(token)
        if len(tokens) != 1 || tokens[0] != strings.ToLower(token) {
            test.Errorf("Token %q was split: %v", token, tokens)
        }
    }
}

Database Tests

Use in-memory SQLite for speed:

func TestChunkStorage(test *testing.T) {
    database, _ := sql.Open("sqlite3", ":memory:")
    defer database.Close()

    store := NewChunkStore(database)
    store.Migrate()

    // Test CRUD operations...
}

HTTP Client Tests

Mock HTTP for Ollama client:

func TestOllamaClient_Embed(test *testing.T) {
    server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
        json.NewEncoder(writer).Encode(map[string]any{
            "embedding": make([]float32, 1536),
        })
    }))
    defer server.Close()

    client := NewOllamaClient(server.URL)
    vector, error := client.Embed(context.Background(), "test text")

    if error != nil {
        test.Fatal(error)
    }
    if len(vector) != 1536 {
        test.Errorf("Expected 1536 dimensions, got %d", len(vector))
    }
}

Integration Tests

Integration tests live in /integration with task-oriented naming.

Naming Convention

Name files by the task being tested, not by feature:

integration/
├── index_documentation_from_files_test.go    ← "Index documentation from files"
├── get_documentation_from_github_test.go     ← "Get documentation from GitHub"
├── search_documentation_test.go              ← "Search documentation"
└── activate_license_test.go                  ← "Activate license"

Structure

Each file tests a complete user task end-to-end:

package integration

// Integration tests for: <task description>
//
// Pipeline: <step 1> → <step 2> → <step 3>
//
// Parent issue: DOC-XX (Step N: Title)

func Test<Task>_FullPipeline(test *testing.T) {
    // Setup temp environment
    // Execute full pipeline
    // Verify end state
}

func Test<Task>_<SpecificBehavior>(test *testing.T) {
    // Test specific aspect of the task
}

Example

// integration/index_documentation_from_files_test.go

package integration

// Integration tests for: indexing documentation from local files
//
// Pipeline: markdown files → normalize → parse → chunk → embed → index
//
// Parent issue: DOC-7 (Step 2: Index Command)

func TestIndexDocumentation_FullPipeline(test *testing.T) {
    // 1. Create temp docs directory with test markdown
    // 2. Mock Ollama server
    // 3. Run: find files → normalize → parse → chunk → embed → index
    // 4. Verify: chunks in DB, vectors.gob exists, bm25.gob exists
    // 5. Verify: search returns expected results
}

func TestIndexDocumentation_DevTokensSearchable(test *testing.T) {
    // Verify $state, on:click, +page.svelte are searchable
}

func TestIndexDocumentation_IncrementalByContentHash(test *testing.T) {
    // Verify content_hash skips re-embedding
}

When to Create Integration Tests

Create integration tests when completing a parent issue that represents a user-facing task:

DOC-7 (Step 2: Index Command) → index_documentation_from_files_test.go
DOC-X (Step 1: Update Command) → get_documentation_from_github_test.go
DOC-Y (Step 3: Search Command) → search_documentation_test.go

Test Commands

# Run all tests (unit + integration) via mage
mage test

# Run only unit tests
go test ./internal/...

# Run only integration tests
go test ./integration/...

# Run with verbose output
go test -v ./internal/indexer/

# Run specific test
go test -run TestNormalize ./internal/indexer/

# Run specific integration test
go test -run TestIndexDocumentation ./integration/

Coverage

Aim for high coverage on core logic:

indexer/normalize.go - 90%+
indexer/parse.go - 85%+
indexer/chunk.go - 85%+
search/bm25.go - 80%+ (tokenizer especially)
search/vector.go - 70%+

Skip coverage on:

CLI wiring (hard to unit test, covered by integration)
Simple CRUD wrappers

FilesExpand file tree

SKILL.md

Latest commit

History