Skip to content

Commit 86990d9

Browse files
committed
chore: doc and oak updates
- Introduced a new document for system prompt design outlining effective prompt anatomy, including role, constraints, task instructions, output format, and examples. - Added context engineering theory to project governance documentation for better rule writing guidance. - Updated .oak configuration to include context-engineering skill. - Created a changelog to document notable changes and updates. - Enhanced contributing guidelines with detailed development setup and PR workflow. - Added a Quick Start Guide for new users to facilitate installation and setup. - Improved README with links to Quick Start, Contributing, and Security policy. - Established a Security Policy for responsible vulnerability reporting and outlined security practices.
1 parent eec03d1 commit 86990d9

52 files changed

Lines changed: 10016 additions & 80 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/skills/codebase-intelligence/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ sqlite3 -readonly -header -column .oak/ci/activities.db "YOUR QUERY HERE"
130130
| `agent_runs` | CI agent executions | `agent_name`, `task`, `status`, `result`, `cost_usd`, `turns_used` |
131131
| `session_link_events` | Session linking analytics | `session_id`, `event_type`, `old_parent_id`, `new_parent_id` |
132132
| `session_relationships` | Semantic session relationships | `session_a_id`, `session_b_id`, `relationship_type`, `similarity_score` |
133-
| `agent_schedules` | Cron scheduling state | `task_name`, `cron_expression`, `enabled`, `last_run_at`, `next_run_at` |
133+
| `agent_schedules` | Cron scheduling state | `task_name`, `cron_expression`, `enabled`, `additional_prompt`, `last_run_at`, `next_run_at` |
134134
| `resolution_events` | Cross-machine resolution propagation | `observation_id`, `action`, `source_machine_id`, `applied`, `content_hash` |
135135
<!-- END GENERATED CORE TABLES -->
136136

.claude/skills/codebase-intelligence/references/schema.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Complete DDL for the Oak CI SQLite database at `.oak/ci/activities.db`.
44

5-
Current schema version: **3**
5+
Current schema version: **4**
66

77
## memory_observations
88

@@ -228,6 +228,7 @@ CREATE TABLE IF NOT EXISTS agent_schedules (
228228
cron_expression TEXT, -- Cron expression (e.g., '0 0 * * MON')
229229
description TEXT, -- Human-readable schedule description
230230
trigger_type TEXT DEFAULT 'cron', -- 'cron' or 'manual' (future: 'git_commit', 'file_change')
231+
additional_prompt TEXT, -- Persistent assignment prepended to task on each run
231232

232233
-- Runtime state
233234
last_run_at TEXT,

.claude/skills/context-engineering/SKILL.md

Lines changed: 311 additions & 0 deletions
Large diffs are not rendered by default.

.claude/skills/context-engineering/references/agent-context-patterns.md

Lines changed: 508 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 331 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,331 @@
1+
# Before/After Examples
2+
3+
Concrete transformations showing how context engineering techniques improve real prompts. Each example: **Before** (problematic), **Diagnosis** (what's wrong), **Techniques Applied**, **After** (improved).
4+
5+
---
6+
7+
## 1. Vague Prompt to Structured Prompt
8+
9+
### Before
10+
```
11+
Help me with this code. It's not working right and I need it fixed.
12+
```
13+
14+
### Diagnosis
15+
- "Not working" gives zero actionable detail -- no code, no error, no expected behavior
16+
- No output format specified
17+
18+
### Techniques Applied
19+
- **Be Clear and Direct**: state the exact problem and expected vs actual behavior
20+
- **XML Tags**: separate code, error, and instructions
21+
22+
### After
23+
```xml
24+
<code language="python">
25+
def calculate_discount(price, tier):
26+
if tier == "gold": return price * 0.8
27+
if tier == "silver": return price * 0.9
28+
return price
29+
</code>
30+
31+
<error>calculate_discount(100, "Gold") returns 100 instead of 80.</error>
32+
33+
<instructions>
34+
1. Identify why the function fails for "Gold" (capital G)
35+
2. Fix the bug
36+
3. Add handling for invalid tier values (return price unchanged, log a warning)
37+
</instructions>
38+
39+
<output_format>Root cause (one sentence), fixed code, test cases.</output_format>
40+
```
41+
42+
**Why it works:** The model knows exactly what is broken, what the expected behavior is, and what to produce. No ambiguity about scope or success criteria.
43+
44+
---
45+
46+
## 2. Wall of Text to XML-Structured Context
47+
48+
### Before
49+
```
50+
Here's my API spec and some error logs and the database schema. The API
51+
spec says we have a /users endpoint that accepts POST with name and email.
52+
The email should be unique. Here are the error logs: [2024-01-15 14:23:01]
53+
IntegrityError: duplicate key violates "users_email_key" Request: POST
54+
/users {"name": "Jane", "email": "jane@test.com"} And the schema is
55+
CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT NOT NULL, email TEXT
56+
NOT NULL UNIQUE). Why am I getting these errors?
57+
```
58+
59+
### Diagnosis
60+
- Three data sources in one paragraph with no separation
61+
- Question buried at the end -- model reads data without knowing the goal
62+
63+
### Techniques Applied
64+
- **XML Tags**: each data source in its own labeled section
65+
- **Long Context Tips**: front-load the question
66+
67+
### After
68+
```xml
69+
<question>
70+
Why do duplicate key errors occur on POST /users? Provide: root cause, fix, error response.
71+
</question>
72+
73+
<error_logs>
74+
[2024-01-15 14:23:01] IntegrityError: duplicate key violates "users_email_key"
75+
Request: POST /users {"name": "Jane", "email": "jane@test.com"}
76+
[2024-01-15 14:25:12] IntegrityError: duplicate key violates "users_email_key"
77+
Request: POST /users {"name": "John", "email": "jane@test.com"}
78+
</error_logs>
79+
80+
<database_schema>
81+
CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT NOT NULL, email TEXT NOT NULL UNIQUE);
82+
</database_schema>
83+
84+
<api_spec>POST /users — Body: { name, email } — email must be unique</api_spec>
85+
```
86+
87+
**Why it works:** Each data source is labeled. The question is front-loaded so the model reads data with the goal already in mind.
88+
89+
---
90+
91+
## 3. Bloated System Prompt to Altitude-Optimized
92+
93+
### Before (22 generic directives)
94+
```
95+
You are an AI assistant. Be helpful, harmless, honest. Be polite.
96+
Use clear language. Be concise but thorough. Prioritize accuracy.
97+
Always cite sources. Be creative when appropriate. Ask clarifying
98+
questions. Be empathetic. Use examples. Stay on topic. [... 10 more ...]
99+
```
100+
101+
### Diagnosis
102+
- None specific enough to change behavior -- "be helpful" is default
103+
- Contradictions: "be concise" vs "be thorough"
104+
- No role, no domain, no output conventions
105+
106+
### Techniques Applied
107+
- **Altitude optimization**: raise to principled guidance
108+
- **System prompt design**: define role with domain expertise
109+
110+
### After
111+
```xml
112+
<role>
113+
Code reviewer. Domain: Django REST APIs with PostgreSQL. Senior level.
114+
</role>
115+
116+
<constraints>
117+
- Flag security issues as CRITICAL regardless of other priorities
118+
- If a change affects schema, always mention migration safety
119+
- When trade-offs exist, state both options and recommend one
120+
- Say "I'm not sure" rather than guessing on library-specific behavior
121+
</constraints>
122+
123+
<output_conventions>
124+
- Reference file:line_number for each finding
125+
- Severity: CRITICAL, WARNING, SUGGESTION
126+
- Include corrected code for CRITICAL and WARNING items
127+
</output_conventions>
128+
```
129+
130+
**Why it works:** 22 vague directives became 7 specific rules. Generic advice is gone; domain-specific guidance takes its place.
131+
132+
---
133+
134+
## 4. Context-Stuffed Agent to Compress and Isolate
135+
136+
### Before
137+
```
138+
System prompt permanently includes:
139+
- Full API docs (8K tokens), database schema (3K), error codes (2K),
140+
coding standards (1.5K), changelog (4K) = ~18,500 tokens always loaded
141+
```
142+
143+
### Diagnosis
144+
- Most reference data used rarely (~5% per conversation)
145+
- 18K tokens permanently occupied; position bias weakens middle sections
146+
- Stale data in the system prompt is worse than no data
147+
148+
### Techniques Applied
149+
- **Isolate**: move reference data behind tool calls
150+
- **Compress**: summarize what remains
151+
- **Select**: retrieve only what the current query needs
152+
153+
### After
154+
```
155+
System prompt (800 tokens): role, principles, tool instructions
156+
157+
Tools:
158+
- search_api_docs(endpoint) -> docs for one endpoint
159+
- query_schema(table_name) -> schema for one table
160+
- lookup_error_code(code) -> one error definition
161+
- get_coding_standards(topic) -> relevant standards
162+
```
163+
164+
**Why it works:** 18,500 to 800 tokens (95% reduction). Data always current (retrieved live). Model attention focused. Cost drops proportionally.
165+
166+
---
167+
168+
## 5. Stateless Agent to Memory-Augmented
169+
170+
### Before
171+
```
172+
User: "Can you help me with the auth module?"
173+
Agent: Starts from scratch. Doesn't know about the circular import
174+
bug fixed last week or the refactor merged yesterday.
175+
```
176+
177+
### Diagnosis
178+
- No persistent memory -- each session reinvents the wheel
179+
- Same gotchas hit repeatedly; user re-explains context every time
180+
181+
### Techniques Applied
182+
- **Write**: store observations as they emerge
183+
- **Select**: retrieve relevant memories at session start
184+
185+
### After
186+
```
187+
Session start: Agent receives injected context:
188+
- Recent session summaries, active gotchas, unresolved observations
189+
190+
During work: Agent stores learnings:
191+
oak_remember("Auth circular import caused by module-level import
192+
of UserModel. Fixed by moving inside function.",
193+
memory_type="bug_fix", context="src/services/auth_service.py")
194+
195+
Next session: "Help with the auth module?"
196+
Agent already knows about the circular import and last week's refactor.
197+
```
198+
199+
**Why it works:** The agent compounds knowledge across sessions. Gotchas caught once stay caught.
200+
201+
---
202+
203+
## 6. Poor Few-Shot to Diverse Multishot
204+
205+
### Before (2 trivial examples)
206+
```
207+
Convert descriptions to SQL queries.
208+
Example: "Get all users" -> SELECT * FROM users;
209+
Example: "Get active users" -> SELECT * FROM users WHERE active = true;
210+
Now convert: "Get the top 5 customers by total spending last quarter"
211+
```
212+
213+
### Diagnosis
214+
- 2 examples, both trivially simple (single table, no joins)
215+
- Actual task requires JOIN, GROUP BY, ORDER BY, LIMIT, date filtering
216+
- No schema -- model guesses table and column names
217+
218+
### Techniques Applied
219+
- **Multishot examples**: 5 diverse examples, simple to complex
220+
- **XML Tags**: structure examples and provide schema
221+
222+
### After
223+
```xml
224+
Convert natural language to PostgreSQL using this schema:
225+
226+
<schema>
227+
users(id, name, email, created_at)
228+
orders(id, user_id, total_amount, created_at, status)
229+
products(id, name, price, category)
230+
order_items(order_id, product_id, quantity)
231+
</schema>
232+
233+
<examples>
234+
<example>
235+
<input>Get all users</input>
236+
<output>SELECT * FROM users;</output>
237+
</example>
238+
<example>
239+
<input>Active orders with user names</input>
240+
<output>SELECT u.name, o.id, o.total_amount
241+
FROM orders o JOIN users u ON u.id = o.user_id WHERE o.status = 'active';</output>
242+
</example>
243+
<example>
244+
<input>Count orders per user, highest first</input>
245+
<output>SELECT u.name, COUNT(o.id) AS order_count
246+
FROM users u LEFT JOIN orders o ON o.user_id = u.id
247+
GROUP BY u.id, u.name ORDER BY order_count DESC;</output>
248+
</example>
249+
<example>
250+
<input>Revenue by product category this year</input>
251+
<output>SELECT p.category, SUM(oi.quantity * p.price) AS revenue
252+
FROM order_items oi JOIN products p ON p.id = oi.product_id
253+
JOIN orders o ON o.id = oi.order_id
254+
WHERE o.created_at >= DATE_TRUNC('year', CURRENT_DATE)
255+
GROUP BY p.category ORDER BY revenue DESC;</output>
256+
</example>
257+
<example>
258+
<input>Users who never placed an order</input>
259+
<output>SELECT u.name, u.email FROM users u
260+
LEFT JOIN orders o ON o.user_id = u.id WHERE o.id IS NULL;</output>
261+
</example>
262+
</examples>
263+
264+
<input>Get the top 5 customers by total spending last quarter</input>
265+
```
266+
267+
**Why it works:** Examples progress simple to complex, covering JOINs, GROUP BY, LEFT JOIN for negation, date functions. Schema is explicit.
268+
269+
---
270+
271+
## 7. Unconstrained Output to Prefilled and Formatted
272+
273+
### Before
274+
```
275+
Analyze this error log and tell me what happened.
276+
[error log contents]
277+
```
278+
279+
### Diagnosis
280+
- No output structure -- could be a paragraph, list, or narrative
281+
- "What happened" is open-ended; no actionable structure
282+
283+
### Techniques Applied
284+
- **Output format**: define exact structure expected
285+
- **Be Clear and Direct**: bound the scope of analysis
286+
287+
### After
288+
```xml
289+
<instructions>
290+
Analyze this error log. Identify root cause, impact, and fix.
291+
Respond in exactly this JSON format -- no additional commentary.
292+
</instructions>
293+
294+
<error_log>[error log contents]</error_log>
295+
296+
<output_format>
297+
{
298+
"root_cause": {
299+
"error": "specific error message",
300+
"location": "file:line or service",
301+
"explanation": "one sentence"
302+
},
303+
"impact": {
304+
"user_facing": "what the user experienced",
305+
"scope": "affected users/requests if determinable"
306+
},
307+
"fix": {
308+
"immediate": "what to do now",
309+
"preventive": "what to change to prevent recurrence"
310+
}
311+
}
312+
</output_format>
313+
```
314+
315+
**Why it works:** The JSON schema acts as a contract. The model produces parseable output, and the three-part structure ensures analysis is actionable, not just descriptive.
316+
317+
---
318+
319+
## Pattern Summary
320+
321+
| Transformation | Key Technique | Typical Improvement |
322+
|---|---|---|
323+
| Vague to Structured | Clarity + XML tags | Consistent, relevant responses |
324+
| Wall of text to XML sections | XML structure | Better information extraction |
325+
| Low altitude to Right altitude | Altitude optimization | Smaller prompt, better adherence |
326+
| Context-stuffed to Tool-based | Isolate + Select | 90%+ token reduction, fresher data |
327+
| Stateless to Memory-augmented | Write + Select | Compounding effectiveness over time |
328+
| Single example to Diverse multishot | Multishot examples | Edge case handling, format consistency |
329+
| Unconstrained to Formatted | Output format + Prefill | Parseable, actionable output |
330+
331+
Each transformation applies techniques from [Prompt Engineering Foundations](prompt-foundations.md). For system-level patterns, see [System Prompt Design](system-prompt-design.md). For agent-specific strategies, see [Agent Context Patterns](agent-context-patterns.md).

0 commit comments

Comments
 (0)