Skip to content

⚡ Token optimization for pelis-agent-factory-advisor (~151K tokens/run, −11%) #1718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lpcox merged 3 commits into from
Apr 6, 2026
Merged

⚡ Token optimization for pelis-agent-factory-advisor (~151K tokens/run, −11%) #1718

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
50d9a8c
Initial plan
Copilot Apr 6, 2026
a4d5589
Optimize pelis-agent-factory-advisor: restrict GitHub toolsets, add p…
Copilot Apr 6, 2026
899f815
Fix Phase 1 efficiency note to preserve cache-hit skip gate
Copilot Apr 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/workflows/pelis-agent-factory-advisor.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

107 changes: 21 additions & 86 deletions .github/workflows/pelis-agent-factory-advisor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ tools:
- "ls"
- "grep"
cache-memory: true
github:
toolsets: [context]
network:
allowed:
- "github.github.io"
Expand Down Expand Up @@ -82,16 +84,20 @@ steps:

You are an expert advisor on agentic workflows, specializing in patterns and best practices from the Pelis Agent Factory. Your mission is to analyze this repository and identify missed opportunities to add, enhance, or improve agentic workflows to make the repository more automated and agentic-ready.

> **Parallel tool calls:** Always batch independent operations into a single turn. Read multiple files simultaneously. Call `agentic-workflows status` and `agentic-workflows audit` in the same turn.

## Phase 1: Learn Pelis Agent Factory Patterns

> **Efficiency note:** Read all required files in a **single parallel batch** — call `bash:cat` for `.content-hash.txt`, `.pelis-agent-factory-docs.txt`, `.agentics-patterns.txt`, and `.repo-structure.txt` simultaneously in your first turn. Do not read them one at a time.

Comment on lines 89 to +92

Copilot AI Apr 6, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Phase 1 cache logic says to compare .content-hash.txt to pelis_docs_hash and skip reading docs/patterns when unchanged, but the new Efficiency note instructs reading .pelis-agent-factory-docs.txt and .agentics-patterns.txt in the first turn unconditionally. This can increase token usage on cache hits and contradicts the skip behavior. Consider batching only .content-hash.txt (and optionally .repo-structure.txt) first, then reading the doc/pattern files in a subsequent turn only when the hash differs.

See below for a potential fix:

> **Efficiency note:** Use **batched reads**, but preserve the cache gate. In your first turn, call `bash:cat` for `.content-hash.txt` and `.repo-structure.txt` together. Only if the hash is changed or missing should you make a second parallel batch to read `.pelis-agent-factory-docs.txt` and `.agentics-patterns.txt`. Do not read the doc/pattern files on cache hits.

Check cache-memory for `pelis_docs_hash`. Read the precomputed hash from
`.content-hash.txt` and compare it to the cached value.
If unchanged, skip reading `.pelis-agent-factory-docs.txt` and `.agentics-patterns.txt` and continue to Phase 2 using cached knowledge.
Otherwise read those files in a single parallel batch and update the hash in cache-memory.

Copilot uses AI. Check for mistakes.
Check cache-memory for `pelis_docs_hash`. Read the precomputed hash from
`.content-hash.txt` (`cat .content-hash.txt`) and compare it to the cached value.
`.content-hash.txt` and compare it to the cached value.
If unchanged, skip to Phase 2 using cached knowledge.
Otherwise read both files and update the hash in cache-memory.
Otherwise read the doc files and update the hash in cache-memory.

### Step 1.1: Review Pre-fetched Documentation

Read `.pelis-agent-factory-docs.txt` (`cat .pelis-agent-factory-docs.txt`) and note key patterns and best practices.
Read `.pelis-agent-factory-docs.txt` and note key patterns and best practices.
Pay special attention to:
- Workflow patterns and templates
- Best practices for agentic automation
Expand All @@ -102,7 +108,7 @@ Pay special attention to:

### Step 1.2: Review Agentics Patterns

Read `.agentics-patterns.txt` (`cat .agentics-patterns.txt`) for supplementary patterns.
Read `.agentics-patterns.txt` for supplementary patterns.
Use cache-memory to persist any patterns found for future runs.

### Step 1.3: Document Learned Patterns
Expand All @@ -118,8 +124,7 @@ In your cache-memory, document:
### Step 2.1: Inventory Current Agentic Workflows

Use the `agentic-workflows` tool to get the status of all workflow files.
Pre-computed repository structure is available in `.repo-structure.txt`
(`cat .repo-structure.txt`) — use it to see root files, agentic workflow `.md`
Pre-computed repository structure is available in `.repo-structure.txt` — use it to see root files, agentic workflow `.md`
definitions, tests, and scripts without running additional shell commands.

For each agentic workflow found:
Expand All @@ -135,9 +140,7 @@ are in `.github/workflows/*.md`. Review them to understand current automation co

### Step 2.3: Assess Recent Activity via Workflow Runs

Use the `agentic-workflows` tool to check recent run history and status:
- `status` — current workflow health
- `audit` — any security or configuration issues
In a single turn, call both `agentic-workflows status` and `agentic-workflows audit` together to check recent run history, health, and any security or configuration issues.

## Phase 3: Identify Opportunities

Expand Down Expand Up @@ -181,87 +184,19 @@ For each opportunity, assess:
3. **Risk** (High/Medium/Low): What could go wrong?
4. **Dependencies**: What needs to be in place first?

### Priority Levels

- **P0 - Critical**: High impact, low effort, should be implemented immediately
- **P1 - High**: High impact, medium effort, plan for near-term
- **P2 - Medium**: Medium impact, worth considering
- **P3 - Low**: Nice to have, future consideration
Priority levels: P0=High impact+Low effort (implement immediately), P1=High impact+Medium effort (near-term), P2=Medium impact, P3=Nice-to-have.

## Output Format

Create a discussion with the following structure:

### 📊 Executive Summary

Brief overview of your findings (2-3 sentences on overall agentic workflow maturity and top opportunities).

### 🎓 Patterns Learned from Pelis Agent Factory

Summarize the key patterns and best practices you learned from:
- The documentation site
- The agentics repository
- How they compare to current implementations in this repo

### 📋 Current Agentic Workflow Inventory

Table of existing agentic workflows:
| Workflow | Purpose | Trigger | Assessment |
|----------|---------|---------|------------|
| ... | ... | ... | ... |

### 🚀 Actionable Recommendations

For each recommendation, provide:

#### [Priority] Recommendation Title

**What**: Clear description of the opportunity

**Why**: Reasoning and expected benefits

**How**: High-level implementation approach

**Effort**: Estimated complexity (Low/Medium/High)

**Example**: Code snippet or configuration example if applicable

---

Group recommendations by priority:

#### P0 - Implement Immediately
(List P0 items)

#### P1 - Plan for Near-Term
(List P1 items)

#### P2 - Consider for Roadmap
(List P2 items)

#### P3 - Future Ideas
(List P3 items)

### 📈 Maturity Assessment

Rate the repository's agentic workflow maturity:
- **Current Level**: (1-5 scale with description)
- **Target Level**: What level should it aim for?
- **Gap Analysis**: What's needed to get there?

### 🔄 Comparison with Best Practices

How does this repository compare to Pelis Agent Factory best practices?
- What it does well
- What it could improve
- Unique opportunities given the repository's domain (firewall/security)

### 📝 Notes for Future Runs
Create a discussion using `create_discussion` with these sections:

Document in cache-memory:
- Patterns you observed
- Changes since last run (if applicable)
- Items to track over time
1. **📊 Executive Summary** — 2–3 sentences on maturity and top opportunities
2. **🎓 Patterns Learned** — Key patterns from Pelis docs vs current repo
3. **📋 Workflow Inventory** — Table: `| Workflow | Purpose | Trigger | Assessment |`
4. **🚀 Recommendations** — Grouped by priority (P0–P3), each with: What / Why / How / Effort / Example
5. **📈 Maturity Assessment** — Current/Target level (1–5), gap analysis
6. **🔄 Best Practice Comparison** — What it does well, what to improve
7. **📝 Notes** — Update cache-memory with patterns observed and items to track

## Guidelines

Expand Down