Skip to content

Reduce Refactoring Scanner AI-credit burn from duplicate-issue lookups#5125

Merged
lpcox merged 3 commits into
mainfrom
copilot/aw-refactoring-opportunity-scanner
Jun 16, 2026
Merged

Reduce Refactoring Scanner AI-credit burn from duplicate-issue lookups#5125
lpcox merged 3 commits into
mainfrom
copilot/aw-refactoring-opportunity-scanner

Conversation

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

The Refactoring Opportunity Scanner was tripping max-ai-credits due to high-cost context during duplicate issue checks, and surfaced an unknown-model pricing failure path. This change narrows Phase 5 data retrieval so duplicate detection runs on lightweight metadata instead of large issue bodies.

  • Prompt hardening for duplicate checks

    • Updated refactoring-scanner.md Phase 5 guidance to require metadata-only issue queries.
    • Explicitly limits query fields to: number, title, state, stateReason, labels, url.
    • Adds a guardrail to not fetch issue bodies unless strictly needed.
  • Compiled workflow metadata sync

    • Updated refactoring-scanner.lock.yml metadata hash to match the prompt body change.
3. Use metadata-only issue queries (number/title/state/stateReason/labels/url). **Do not fetch issue bodies** unless strictly required.

Copilot AI changed the title [WIP] Fix AI credits rate limit in Opportunity Scanner Reduce Refactoring Scanner AI-credit burn from duplicate-issue lookups Jun 16, 2026
Copilot finished work on behalf of lpcox June 16, 2026 19:04
Copilot AI requested a review from lpcox June 16, 2026 19:04
@lpcox lpcox marked this pull request as ready for review June 16, 2026 19:54
Copilot AI review requested due to automatic review settings June 16, 2026 19:54
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 97.27% 97.31% 📈 +0.04%
Statements 97.15% 97.19% 📈 +0.04%
Functions 98.81% 98.81% ➡️ +0.00%
Branches 91.95% 91.98% 📈 +0.03%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/workdir-setup.ts 92.6% → 94.4% (+1.85%) 92.6% → 94.4% (+1.85%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Refactoring Opportunity Scanner workflow prompt to reduce AI-credit usage during duplicate-issue detection by requiring metadata-only issue lookups (instead of fetching full issue bodies), and syncs the compiled lock workflow metadata accordingly.

Changes:

  • Harden Phase 5 guidance to use metadata-only issue queries and avoid fetching issue bodies unless strictly required.
  • Update the compiled refactoring-scanner.lock.yml metadata hash to match the prompt change.
Show a summary per file
File Description
.github/workflows/refactoring-scanner.md Adds explicit guidance to use metadata-only issue fields for duplicate checks to reduce context/credit usage.
.github/workflows/refactoring-scanner.lock.yml Updates the gh-aw metadata hash to reflect the prompt body change.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment thread .github/workflows/refactoring-scanner.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Contribution Check failed. Please review the logs for details.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini completed. All facets verified. 💎

Smoke test completed. Overall status: FAIL due to connectivity issues.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude failed

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

@github-actions

Copy link
Copy Markdown
Contributor

🔥 Smoke Test: Copilot PAT Auth — PASS

Test Result
GitHub MCP connectivity
GitHub.com HTTP (200)
File write/read

Overall: PASS — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

cc @lpcox @Copilot

🔑 PAT report filed by Smoke Copilot PAT

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

Test Results:

  • ✅ MCP connectivity (GitHub API verified)
  • ✅ GitHub.com connectivity (HTTP 200)
  • ✅ File write/read test (agent filesystem)
  • ✅ BYOK inference test (direct mode via api-proxy → api.githubcopilot.com)

Mode: Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy sidecar to api.githubcopilot.com

Overall Status: ✅ PASS


Smoke test executed by @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

Copy link
Copy Markdown
Contributor

🔥 Smoke Test Results — PASS

Test Result
GitHub MCP connectivity
GitHub.com HTTP connectivity
File write/read smoke-test-copilot-27647960798.txt

PR: Reduce Refactoring Scanner AI-credit burn from duplicate-issue lookups
Author: @Copilot | Assignees: @lpcox @Copilot

Overall: ✅ PASS

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

Copy link
Copy Markdown
Contributor

Merged PRs:

  • fix(ci-gate): tag PR owner with instructions
  • docs: sync schemas and specs with source changes
    Results: ✅ GitHub query, ✅ Playwright, ✅ temp file, ✅ discussion, ✅ build
    Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

Copy link
Copy Markdown
Contributor

📡 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario Result
Scenario 1: Module Loading otel.js loads successfully; isEnabled: true; exports: startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled + test helpers
Scenario 2: Test Suite ✅ 59 tests passed, 0 failed (2 suites: otel.test.js, otel-fanout.test.js)
Scenario 3: Env Var Forwarding ⚠️ src/services/api-proxy-service.ts does not yet forward OTEL_EXPORTER_OTLP_ENDPOINT / GITHUB_AW_OTEL_TRACE_ID — expected during development
Scenario 4: Token Tracker Integration onUsage callback present in token-tracker-http.js (OTEL hook point wired)
Scenario 5: OTEL Diagnostics ⚠️ No otel.jsonl span file found — api-proxy OTEL not yet active in sandbox (expected: env vars not forwarded yet)

Summary: Core OTEL module and test suite are fully functional. The only pending item is forwarding OTEL env vars from the CLI into the api-proxy container (api-proxy-service.ts), which is expected during development.

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.13 Python 3.12.3 ❌ No
Node.js v24.16.0 v22.22.3 ❌ No
Go go1.22.12 go1.22.12 ✅ Yes

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot.

Tested by Smoke Chroot

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #5125 ·

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results

  1. GitHub MCP Testing: ✅
    • docs: sync schemas and specs with source changes
    • [Test Coverage] Improve test coverage for network-options, environment-builder, log-parser, workspace-mounts
  2. GitHub.com Connectivity: ❌ (HTTP 400 via Squid)
  3. File Writing Testing: ✅
  4. Bash Tool Testing: ✅

Overall Status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: GitHub Actions Services Connectivity

Check Result
Redis PING ❌ Connection timed out
PostgreSQL pg_isready ❌ No response
PostgreSQL SELECT 1 ❌ No response

host.docker.internal resolves (172.17.0.1) but ports 6379 and 5432 are unreachable from this environment.

Overall: FAIL

🔌 Service connectivity validated by Smoke Services

@github-actions

Copy link
Copy Markdown
Contributor

@lpcox @Copilot
Smoke Test:

  • MCP API: ✅
  • github.com HTTP: ✅
  • File IO: ✅
  • Direct BYOK: ✅
    Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
    Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

Copy link
Copy Markdown
Contributor

@Copilot @lpcox

  • GitHub MCP API: ✅
  • GitHub.com connectivity: ✅
  • Agent file I/O: ✅
  • BYOK inference: ✅
    Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
    Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@lpcox lpcox merged commit b8fc6d4 into main Jun 16, 2026
83 of 86 checks passed
@lpcox lpcox deleted the copilot/aw-refactoring-opportunity-scanner branch June 16, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants