Skip to content

Commit 8f239d9

Browse files
Tobias-Benedikt Blaskclaude
andcommitted
feat: Add orchestration audit trail logging (v5.2.0)
Add structured logging instructions to the agent prompt and all 7 skill engines. Every pipeline run now produces an orchestration_log.md that records timestamps, actors, inputs/outputs, and quality gate decisions. This operationalizes the Orchestration Trace concept for publishable human-AI accountability documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 857d9ae commit 8f239d9

9 files changed

Lines changed: 131 additions & 1 deletion

File tree

.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "open-academic-paper-machine",
33
"description": "Open Academic Paper Machine — Autonomous academic paper production system. Uses MCP academic-search tools (Semantic Scholar, OpenAlex, CrossRef, arXiv) for literature search, snowballing, and export. PaperBanana MCP for figure generation. Theory framing, methodology, writing, and citation verification engines with fill-in-the-blank templates. One command (/write-paper) triggers the full 7-phase pipeline from topic to complete first draft with LaTeX, PDF output, and citation verification. /verify-citations for standalone citation checking.",
4-
"version": "5.1.0",
4+
"version": "5.2.0",
55
"author": { "name": "Prof. Dr. Tobias Blask" },
66
"license": "MIT",
77
"mcpServers": {

agents/paper-machine.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,73 @@ theory selection, gap formulation, method design, and full-text drafting.
2424
3. **Checkpoint, don't block.** Present work for approval, then continue. Don't wait for permission to start.
2525
4. **Be explicit about decisions.** State what you chose and why. Let the user override.
2626
5. **Save everything to files.** Every phase produces saved artifacts the user can review.
27+
6. **Log everything to the orchestration log.** Every phase transition, quality gate decision, and human override is recorded for transparency and auditability.
28+
29+
## Orchestration Log
30+
31+
At the very start of a pipeline run, **create `outputs/orchestration_log.md`** with the following header:
32+
33+
```markdown
34+
# Orchestration Log
35+
**Paper:** [title or topic from user input]
36+
**Started:** [current date and time, ISO 8601]
37+
**Orchestrator:** [user, if known]
38+
**AI Agent:** Claude (via Open Paper Machine)
39+
40+
---
41+
42+
This log records every significant interaction between the human orchestrator and the AI agent during the paper production process. It is designed for publication alongside the manuscript (e.g., on GitHub) to make the human-AI division of labor transparent and auditable.
43+
44+
---
45+
```
46+
47+
### Logging Rules
48+
49+
**BEFORE each checkpoint**, append to `outputs/orchestration_log.md`:
50+
51+
```markdown
52+
## Phase [N]: [Phase Name]
53+
**Timestamp:** [current date/time]
54+
**Actor:** AI Agent
55+
**Action:** [brief description of what was produced]
56+
**Key metrics:** [papers found / words written / sections completed / etc.]
57+
**Output artifacts:** [list of files saved]
58+
```
59+
60+
**AFTER the user responds to a checkpoint**, append:
61+
62+
```markdown
63+
**Quality Gate Decision:** [Approved / Redirected / Rejected]
64+
**Orchestrator Feedback:** "[verbatim quote of user's response, or 'No objection — auto-proceeded']"
65+
**Scope Changes:** [any changes to direction, if applicable, or 'None']
66+
```
67+
68+
**When the user overrides or redirects mid-phase**, append:
69+
70+
```markdown
71+
### Mid-Phase Intervention
72+
**Timestamp:** [current date/time]
73+
**Actor:** Human Orchestrator
74+
**Action:** [Override / Redirect / Additional instruction]
75+
**Content:** "[verbatim quote of user instruction]"
76+
**Agent Response:** [what the agent did in response]
77+
```
78+
79+
**At pipeline completion**, append a summary:
80+
81+
```markdown
82+
---
83+
84+
## Pipeline Summary
85+
**Completed:** [current date/time]
86+
**Total phases executed:** [N]
87+
**Human interventions:** [N] (quality gates: [N], mid-phase redirects: [N])
88+
**Key decisions by orchestrator:**
89+
- [decision 1]
90+
- [decision 2]
91+
- [...]
92+
**Final artifacts:** [list all output files]
93+
```
2794

2895
---
2996

skills/figure-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@ description: >
77
Supports methodology diagrams, statistical plots, and comparative evaluation.
88
---
99

10+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
11+
> ```
12+
> ### Skill Activation: Figure Engine
13+
> **Timestamp:** [current date/time]
14+
> **Actor:** AI Agent (figure-engine)
15+
> **Input:** [brief description of the figure request]
16+
> **Output:** [brief description of what was produced — e.g., "Generated methodology diagram (fig03_methodology.png), 3 iterations"]
17+
> ```
18+
1019
# Figure Engine
1120
1221
## Core Principle

skills/latex-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@ description: >
77
tables, figure environments, and compiled PDF output.
88
---
99

10+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
11+
> ```
12+
> ### Skill Activation: LaTeX Engine
13+
> **Timestamp:** [current date/time]
14+
> **Actor:** AI Agent (latex-engine)
15+
> **Input:** [brief description of the export request]
16+
> **Output:** [brief description of what was produced — e.g., "Compiled paper.tex to PDF (24 pages), no errors"]
17+
> ```
18+
1019
# LaTeX Engine
1120
1221
## Core Principle

skills/literature-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@ description: >
88
APIs first, web search only for verification.
99
---
1010

11+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
12+
> ```
13+
> ### Skill Activation: Literature Engine
14+
> **Timestamp:** [current date/time]
15+
> **Actor:** AI Agent (literature-engine)
16+
> **Input:** [brief description of the search/synthesis request]
17+
> **Output:** [brief description of results — e.g., "47 papers found across 4 databases, deduplicated to 38"]
18+
> ```
19+
1120
# Literature Engine
1221
1322
## Data Source Priority — STRICT

skills/method-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@ description: >
88
and mixed methods.
99
---
1010

11+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
12+
> ```
13+
> ### Skill Activation: Method Engine
14+
> **Timestamp:** [current date/time]
15+
> **Actor:** AI Agent (method-engine)
16+
> **Input:** [brief description of the methodology request]
17+
> **Output:** [brief description of what was produced — e.g., "DSR method section drafted with 3 evaluation criteria"]
18+
> ```
19+
1120
# Method Engine
1221
1322
## Method Selection Guide

skills/theory-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,15 @@ description: >
66
Provides concrete theory-to-paper templates, not abstract advice.
77
---
88

9+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
10+
> ```
11+
> ### Skill Activation: Theory Engine
12+
> **Timestamp:** [current date/time]
13+
> **Actor:** AI Agent (theory-engine)
14+
> **Input:** [brief description of the theory selection/gap formulation request]
15+
> **Output:** [brief description of what was produced — e.g., "Selected sociotechnical systems theory, drafted theoretical framing (800 words)"]
16+
> ```
17+
918
# Theory Engine
1019
1120
## Purpose

skills/verification-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,15 @@ description: >
99
MISMATCH, UNVERIFIABLE, NOT FOUND).
1010
---
1111

12+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
13+
> ```
14+
> ### Skill Activation: Verification Engine
15+
> **Timestamp:** [current date/time]
16+
> **Actor:** AI Agent (verification-engine)
17+
> **Input:** [brief description of the verification request]
18+
> **Output:** [brief description of results — e.g., "Verified 42 citations: 35 VERIFIED, 5 PLAUSIBLE, 2 MISMATCH"]
19+
> ```
20+
1221
# Verification Engine
1322
1423
## Core Principle

skills/writing-engine/SKILL.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@ description: >
88
conference papers (ICIS, ECIS, WI), and working papers.
99
---
1010

11+
> **Orchestration Log**: When this skill is activated, append a log entry to `outputs/orchestration_log.md`:
12+
> ```
13+
> ### Skill Activation: Writing Engine
14+
> **Timestamp:** [current date/time]
15+
> **Actor:** AI Agent (writing-engine)
16+
> **Input:** [brief description of what the user asked to write/draft/revise]
17+
> **Output:** [brief description of what was produced — e.g., "Complete draft of Introduction (1,200 words)"]
18+
> ```
19+
1120
# Writing Engine
1221
1322
## Core Principle

0 commit comments

Comments
 (0)