|
| 1 | +# Before/After Examples |
| 2 | + |
| 3 | +Concrete transformations showing how context engineering techniques improve real prompts. Each example: **Before** (problematic), **Diagnosis** (what's wrong), **Techniques Applied**, **After** (improved). |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## 1. Vague Prompt to Structured Prompt |
| 8 | + |
| 9 | +### Before |
| 10 | +``` |
| 11 | +Help me with this code. It's not working right and I need it fixed. |
| 12 | +``` |
| 13 | + |
| 14 | +### Diagnosis |
| 15 | +- "Not working" gives zero actionable detail -- no code, no error, no expected behavior |
| 16 | +- No output format specified |
| 17 | + |
| 18 | +### Techniques Applied |
| 19 | +- **Be Clear and Direct**: state the exact problem and expected vs actual behavior |
| 20 | +- **XML Tags**: separate code, error, and instructions |
| 21 | + |
| 22 | +### After |
| 23 | +```xml |
| 24 | +<code language="python"> |
| 25 | +def calculate_discount(price, tier): |
| 26 | + if tier == "gold": return price * 0.8 |
| 27 | + if tier == "silver": return price * 0.9 |
| 28 | + return price |
| 29 | +</code> |
| 30 | + |
| 31 | +<error>calculate_discount(100, "Gold") returns 100 instead of 80.</error> |
| 32 | + |
| 33 | +<instructions> |
| 34 | +1. Identify why the function fails for "Gold" (capital G) |
| 35 | +2. Fix the bug |
| 36 | +3. Add handling for invalid tier values (return price unchanged, log a warning) |
| 37 | +</instructions> |
| 38 | + |
| 39 | +<output_format>Root cause (one sentence), fixed code, test cases.</output_format> |
| 40 | +``` |
| 41 | + |
| 42 | +**Why it works:** The model knows exactly what is broken, what the expected behavior is, and what to produce. No ambiguity about scope or success criteria. |
| 43 | + |
| 44 | +--- |
| 45 | + |
| 46 | +## 2. Wall of Text to XML-Structured Context |
| 47 | + |
| 48 | +### Before |
| 49 | +``` |
| 50 | +Here's my API spec and some error logs and the database schema. The API |
| 51 | +spec says we have a /users endpoint that accepts POST with name and email. |
| 52 | +The email should be unique. Here are the error logs: [2024-01-15 14:23:01] |
| 53 | +IntegrityError: duplicate key violates "users_email_key" Request: POST |
| 54 | +/users {"name": "Jane", "email": "jane@test.com"} And the schema is |
| 55 | +CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT NOT NULL, email TEXT |
| 56 | +NOT NULL UNIQUE). Why am I getting these errors? |
| 57 | +``` |
| 58 | + |
| 59 | +### Diagnosis |
| 60 | +- Three data sources in one paragraph with no separation |
| 61 | +- Question buried at the end -- model reads data without knowing the goal |
| 62 | + |
| 63 | +### Techniques Applied |
| 64 | +- **XML Tags**: each data source in its own labeled section |
| 65 | +- **Long Context Tips**: front-load the question |
| 66 | + |
| 67 | +### After |
| 68 | +```xml |
| 69 | +<question> |
| 70 | +Why do duplicate key errors occur on POST /users? Provide: root cause, fix, error response. |
| 71 | +</question> |
| 72 | + |
| 73 | +<error_logs> |
| 74 | +[2024-01-15 14:23:01] IntegrityError: duplicate key violates "users_email_key" |
| 75 | + Request: POST /users {"name": "Jane", "email": "jane@test.com"} |
| 76 | +[2024-01-15 14:25:12] IntegrityError: duplicate key violates "users_email_key" |
| 77 | + Request: POST /users {"name": "John", "email": "jane@test.com"} |
| 78 | +</error_logs> |
| 79 | + |
| 80 | +<database_schema> |
| 81 | +CREATE TABLE users (id SERIAL PRIMARY KEY, name TEXT NOT NULL, email TEXT NOT NULL UNIQUE); |
| 82 | +</database_schema> |
| 83 | + |
| 84 | +<api_spec>POST /users — Body: { name, email } — email must be unique</api_spec> |
| 85 | +``` |
| 86 | + |
| 87 | +**Why it works:** Each data source is labeled. The question is front-loaded so the model reads data with the goal already in mind. |
| 88 | + |
| 89 | +--- |
| 90 | + |
| 91 | +## 3. Bloated System Prompt to Altitude-Optimized |
| 92 | + |
| 93 | +### Before (22 generic directives) |
| 94 | +``` |
| 95 | +You are an AI assistant. Be helpful, harmless, honest. Be polite. |
| 96 | +Use clear language. Be concise but thorough. Prioritize accuracy. |
| 97 | +Always cite sources. Be creative when appropriate. Ask clarifying |
| 98 | +questions. Be empathetic. Use examples. Stay on topic. [... 10 more ...] |
| 99 | +``` |
| 100 | + |
| 101 | +### Diagnosis |
| 102 | +- None specific enough to change behavior -- "be helpful" is default |
| 103 | +- Contradictions: "be concise" vs "be thorough" |
| 104 | +- No role, no domain, no output conventions |
| 105 | + |
| 106 | +### Techniques Applied |
| 107 | +- **Altitude optimization**: raise to principled guidance |
| 108 | +- **System prompt design**: define role with domain expertise |
| 109 | + |
| 110 | +### After |
| 111 | +```xml |
| 112 | +<role> |
| 113 | +Code reviewer. Domain: Django REST APIs with PostgreSQL. Senior level. |
| 114 | +</role> |
| 115 | + |
| 116 | +<constraints> |
| 117 | +- Flag security issues as CRITICAL regardless of other priorities |
| 118 | +- If a change affects schema, always mention migration safety |
| 119 | +- When trade-offs exist, state both options and recommend one |
| 120 | +- Say "I'm not sure" rather than guessing on library-specific behavior |
| 121 | +</constraints> |
| 122 | + |
| 123 | +<output_conventions> |
| 124 | +- Reference file:line_number for each finding |
| 125 | +- Severity: CRITICAL, WARNING, SUGGESTION |
| 126 | +- Include corrected code for CRITICAL and WARNING items |
| 127 | +</output_conventions> |
| 128 | +``` |
| 129 | + |
| 130 | +**Why it works:** 22 vague directives became 7 specific rules. Generic advice is gone; domain-specific guidance takes its place. |
| 131 | + |
| 132 | +--- |
| 133 | + |
| 134 | +## 4. Context-Stuffed Agent to Compress and Isolate |
| 135 | + |
| 136 | +### Before |
| 137 | +``` |
| 138 | +System prompt permanently includes: |
| 139 | +- Full API docs (8K tokens), database schema (3K), error codes (2K), |
| 140 | + coding standards (1.5K), changelog (4K) = ~18,500 tokens always loaded |
| 141 | +``` |
| 142 | + |
| 143 | +### Diagnosis |
| 144 | +- Most reference data used rarely (~5% per conversation) |
| 145 | +- 18K tokens permanently occupied; position bias weakens middle sections |
| 146 | +- Stale data in the system prompt is worse than no data |
| 147 | + |
| 148 | +### Techniques Applied |
| 149 | +- **Isolate**: move reference data behind tool calls |
| 150 | +- **Compress**: summarize what remains |
| 151 | +- **Select**: retrieve only what the current query needs |
| 152 | + |
| 153 | +### After |
| 154 | +``` |
| 155 | +System prompt (800 tokens): role, principles, tool instructions |
| 156 | +
|
| 157 | +Tools: |
| 158 | +- search_api_docs(endpoint) -> docs for one endpoint |
| 159 | +- query_schema(table_name) -> schema for one table |
| 160 | +- lookup_error_code(code) -> one error definition |
| 161 | +- get_coding_standards(topic) -> relevant standards |
| 162 | +``` |
| 163 | + |
| 164 | +**Why it works:** 18,500 to 800 tokens (95% reduction). Data always current (retrieved live). Model attention focused. Cost drops proportionally. |
| 165 | + |
| 166 | +--- |
| 167 | + |
| 168 | +## 5. Stateless Agent to Memory-Augmented |
| 169 | + |
| 170 | +### Before |
| 171 | +``` |
| 172 | +User: "Can you help me with the auth module?" |
| 173 | +Agent: Starts from scratch. Doesn't know about the circular import |
| 174 | + bug fixed last week or the refactor merged yesterday. |
| 175 | +``` |
| 176 | + |
| 177 | +### Diagnosis |
| 178 | +- No persistent memory -- each session reinvents the wheel |
| 179 | +- Same gotchas hit repeatedly; user re-explains context every time |
| 180 | + |
| 181 | +### Techniques Applied |
| 182 | +- **Write**: store observations as they emerge |
| 183 | +- **Select**: retrieve relevant memories at session start |
| 184 | + |
| 185 | +### After |
| 186 | +``` |
| 187 | +Session start: Agent receives injected context: |
| 188 | + - Recent session summaries, active gotchas, unresolved observations |
| 189 | +
|
| 190 | +During work: Agent stores learnings: |
| 191 | + oak_remember("Auth circular import caused by module-level import |
| 192 | + of UserModel. Fixed by moving inside function.", |
| 193 | + memory_type="bug_fix", context="src/services/auth_service.py") |
| 194 | +
|
| 195 | +Next session: "Help with the auth module?" |
| 196 | + Agent already knows about the circular import and last week's refactor. |
| 197 | +``` |
| 198 | + |
| 199 | +**Why it works:** The agent compounds knowledge across sessions. Gotchas caught once stay caught. |
| 200 | + |
| 201 | +--- |
| 202 | + |
| 203 | +## 6. Poor Few-Shot to Diverse Multishot |
| 204 | + |
| 205 | +### Before (2 trivial examples) |
| 206 | +``` |
| 207 | +Convert descriptions to SQL queries. |
| 208 | +Example: "Get all users" -> SELECT * FROM users; |
| 209 | +Example: "Get active users" -> SELECT * FROM users WHERE active = true; |
| 210 | +Now convert: "Get the top 5 customers by total spending last quarter" |
| 211 | +``` |
| 212 | + |
| 213 | +### Diagnosis |
| 214 | +- 2 examples, both trivially simple (single table, no joins) |
| 215 | +- Actual task requires JOIN, GROUP BY, ORDER BY, LIMIT, date filtering |
| 216 | +- No schema -- model guesses table and column names |
| 217 | + |
| 218 | +### Techniques Applied |
| 219 | +- **Multishot examples**: 5 diverse examples, simple to complex |
| 220 | +- **XML Tags**: structure examples and provide schema |
| 221 | + |
| 222 | +### After |
| 223 | +```xml |
| 224 | +Convert natural language to PostgreSQL using this schema: |
| 225 | + |
| 226 | +<schema> |
| 227 | +users(id, name, email, created_at) |
| 228 | +orders(id, user_id, total_amount, created_at, status) |
| 229 | +products(id, name, price, category) |
| 230 | +order_items(order_id, product_id, quantity) |
| 231 | +</schema> |
| 232 | + |
| 233 | +<examples> |
| 234 | +<example> |
| 235 | +<input>Get all users</input> |
| 236 | +<output>SELECT * FROM users;</output> |
| 237 | +</example> |
| 238 | +<example> |
| 239 | +<input>Active orders with user names</input> |
| 240 | +<output>SELECT u.name, o.id, o.total_amount |
| 241 | +FROM orders o JOIN users u ON u.id = o.user_id WHERE o.status = 'active';</output> |
| 242 | +</example> |
| 243 | +<example> |
| 244 | +<input>Count orders per user, highest first</input> |
| 245 | +<output>SELECT u.name, COUNT(o.id) AS order_count |
| 246 | +FROM users u LEFT JOIN orders o ON o.user_id = u.id |
| 247 | +GROUP BY u.id, u.name ORDER BY order_count DESC;</output> |
| 248 | +</example> |
| 249 | +<example> |
| 250 | +<input>Revenue by product category this year</input> |
| 251 | +<output>SELECT p.category, SUM(oi.quantity * p.price) AS revenue |
| 252 | +FROM order_items oi JOIN products p ON p.id = oi.product_id |
| 253 | +JOIN orders o ON o.id = oi.order_id |
| 254 | +WHERE o.created_at >= DATE_TRUNC('year', CURRENT_DATE) |
| 255 | +GROUP BY p.category ORDER BY revenue DESC;</output> |
| 256 | +</example> |
| 257 | +<example> |
| 258 | +<input>Users who never placed an order</input> |
| 259 | +<output>SELECT u.name, u.email FROM users u |
| 260 | +LEFT JOIN orders o ON o.user_id = u.id WHERE o.id IS NULL;</output> |
| 261 | +</example> |
| 262 | +</examples> |
| 263 | + |
| 264 | +<input>Get the top 5 customers by total spending last quarter</input> |
| 265 | +``` |
| 266 | + |
| 267 | +**Why it works:** Examples progress simple to complex, covering JOINs, GROUP BY, LEFT JOIN for negation, date functions. Schema is explicit. |
| 268 | + |
| 269 | +--- |
| 270 | + |
| 271 | +## 7. Unconstrained Output to Prefilled and Formatted |
| 272 | + |
| 273 | +### Before |
| 274 | +``` |
| 275 | +Analyze this error log and tell me what happened. |
| 276 | +[error log contents] |
| 277 | +``` |
| 278 | + |
| 279 | +### Diagnosis |
| 280 | +- No output structure -- could be a paragraph, list, or narrative |
| 281 | +- "What happened" is open-ended; no actionable structure |
| 282 | + |
| 283 | +### Techniques Applied |
| 284 | +- **Output format**: define exact structure expected |
| 285 | +- **Be Clear and Direct**: bound the scope of analysis |
| 286 | + |
| 287 | +### After |
| 288 | +```xml |
| 289 | +<instructions> |
| 290 | +Analyze this error log. Identify root cause, impact, and fix. |
| 291 | +Respond in exactly this JSON format -- no additional commentary. |
| 292 | +</instructions> |
| 293 | + |
| 294 | +<error_log>[error log contents]</error_log> |
| 295 | + |
| 296 | +<output_format> |
| 297 | +{ |
| 298 | + "root_cause": { |
| 299 | + "error": "specific error message", |
| 300 | + "location": "file:line or service", |
| 301 | + "explanation": "one sentence" |
| 302 | + }, |
| 303 | + "impact": { |
| 304 | + "user_facing": "what the user experienced", |
| 305 | + "scope": "affected users/requests if determinable" |
| 306 | + }, |
| 307 | + "fix": { |
| 308 | + "immediate": "what to do now", |
| 309 | + "preventive": "what to change to prevent recurrence" |
| 310 | + } |
| 311 | +} |
| 312 | +</output_format> |
| 313 | +``` |
| 314 | + |
| 315 | +**Why it works:** The JSON schema acts as a contract. The model produces parseable output, and the three-part structure ensures analysis is actionable, not just descriptive. |
| 316 | + |
| 317 | +--- |
| 318 | + |
| 319 | +## Pattern Summary |
| 320 | + |
| 321 | +| Transformation | Key Technique | Typical Improvement | |
| 322 | +|---|---|---| |
| 323 | +| Vague to Structured | Clarity + XML tags | Consistent, relevant responses | |
| 324 | +| Wall of text to XML sections | XML structure | Better information extraction | |
| 325 | +| Low altitude to Right altitude | Altitude optimization | Smaller prompt, better adherence | |
| 326 | +| Context-stuffed to Tool-based | Isolate + Select | 90%+ token reduction, fresher data | |
| 327 | +| Stateless to Memory-augmented | Write + Select | Compounding effectiveness over time | |
| 328 | +| Single example to Diverse multishot | Multishot examples | Edge case handling, format consistency | |
| 329 | +| Unconstrained to Formatted | Output format + Prefill | Parseable, actionable output | |
| 330 | + |
| 331 | +Each transformation applies techniques from [Prompt Engineering Foundations](prompt-foundations.md). For system-level patterns, see [System Prompt Design](system-prompt-design.md). For agent-specific strategies, see [Agent Context Patterns](agent-context-patterns.md). |
0 commit comments