feat(auth0): fold tenant audit (CheckMate) into the unified skill#141
Open
frederikprijck wants to merge 3 commits into
Open
feat(auth0): fold tenant audit (CheckMate) into the unified skill#141frederikprijck wants to merge 3 commits into
frederikprijck wants to merge 3 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds an Auth0 tenant audit capability using CheckMate, including CLI bootstrap and remediation guidance, Markdown/HTML/PDF report generation, pricing references, PDF renderer fallbacks, and routing and behavioral evaluation cases. ChangesAuth0 tenant audit
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant Auth0Skill
participant Auth0CLI
participant CheckMate
participant ReportGenerator
Operator->>Auth0Skill: Request tenant audit
Auth0Skill->>Auth0CLI: Bootstrap or reuse credentials
Auth0CLI-->>CheckMate: Provide tenant and M2M credentials
CheckMate-->>ReportGenerator: Return audit JSON and tenant facts
ReportGenerator-->>Operator: Produce Markdown, HTML, and PDF reports
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
frederikprijck
force-pushed
the
fold-audit-into-router
branch
2 times, most recently
from
July 15, 2026 14:39
11c3e07 to
ba6ea1a
Compare
Add an 'audit' intent to the unified auth0 router that runs an Auth0
CheckMate tenant configuration/security audit, produces a markdown + PDF
report, and applies approved fixes via the Auth0 CLI with per-command
confirmation.
- SKILL.md: Step 1 intent row + Step 4 load block; extend the description.
- references/feature-audit.md: full workflow (CLI bootstrap, M2M app,
company context, CheckMate run, report fusion, gated apply loops).
- references/feature-audit-pricing.md: feature to plan matrix (shared).
- references/feature-audit-remediation.md: command-shape map + safety rules.
- scripts/render_pdf.sh + assets/audit/report-template.{md,html}.
- Routing eval (audit-tenant) + README coverage.
frederikprijck
force-pushed
the
fold-audit-into-router
branch
from
July 20, 2026 20:16
ba6ea1a to
15a8e83
Compare
frederikprijck
marked this pull request as ready for review
July 20, 2026 20:19
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@evals/behavioral/cases/audit.json`:
- Around line 1-25: Update the audit evaluation metadata so both origin_skill
and skill_name use the unified auth0 skill. In the plan recommendation
assertion, replace the exact-cost requirement with the behavior specified by
feature-audit.md: report the plan name only and do not quote or estimate
pricing, including Enterprise pricing.
In `@plugins/auth0/skills/auth0/scripts/render_pdf.sh`:
- Around line 22-24: Validate the output parent directory before the OUTPUT_DIR
assignment changes into it. In render_pdf.sh, check that the directory derived
from OUTPUT is an existing directory; if not, print an actionable error message
identifying the invalid output path and exit non-zero, then preserve the
existing OUTPUT_DIR and OUTPUT_ABS construction for valid paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 680f7b88-5bc2-4a3a-82d2-651697431164
📒 Files selected for processing (10)
evals/behavioral/cases/audit.jsonevals/routing-cases.jsonplugins/auth0/README.mdplugins/auth0/skills/auth0/SKILL.mdplugins/auth0/skills/auth0/assets/audit/report-template.htmlplugins/auth0/skills/auth0/assets/audit/report-template.mdplugins/auth0/skills/auth0/references/feature-audit-pricing.mdplugins/auth0/skills/auth0/references/feature-audit-remediation.mdplugins/auth0/skills/auth0/references/feature-audit.mdplugins/auth0/skills/auth0/scripts/render_pdf.sh
…y assertion - origin_skill/skill_name → auth0 (auth0-checkmate folded into unified skill) - plan recommendation assertion now matches feature-audit.md (plan name only, no price; never quote/estimate Enterprise pricing)
Guard against a non-existent output parent dir before cd — fail with an actionable message and non-zero exit instead of letting cd crash under set -e. Mirrors the existing input-file check.
✅ skillsaw — All checks passedFull report
|
Member
Author
|
@coderabbitai rereview |
|
✅ Action performedReview finished.
|
brth31
reviewed
Jul 27, 2026
| @@ -0,0 +1,588 @@ | |||
| # Auth0 Pricing | |||
Contributor
There was a problem hiding this comment.
how do we keep this updated with the pricing changes on auth0.com/pricing? should we add an instruction for the agent to explicitly check the auth0.com/pricing page? we also support an agent-friendly https://auth0.com/pricing.md page
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Folds the standalone
auth0-checkmateskill (from #127) into the unifiedauth0router skill as a new
auditintent, matching the v2.0 one-skill / flat-reference-poolarchitecture on
main. No behavior is dropped — the CheckMate tenant audit becomes a setof on-demand reference files the router loads when a developer asks to audit a tenant.
The
auditintentAdded to
SKILL.md's intent table and Step 4 loader. When routed, it loads:with the standing rule: apply findings only with per-command confirmation; verify each
change by re-fetch.
How the standalone skill maps onto the fold
auth0-checkmate)SKILL.mdPhases 0–6references/feature-audit.mdSKILL.mdPhase 7 + closingreferences/feature-audit-remediation.mdreferences/auth0-pricing.mdreferences/feature-audit-pricing.mdreferences/checkmate-readme.md+auth0-cli-readme.mdfeature-audit.mdreferences/report-template.{md,html}assets/audit/report-template.{md,html}scripts/render_pdf.shscripts/render_pdf.shtests/evals.json+tests/graders.jsonevals/behavioral/cases/audit.jsonEverything material is preserved: all seven workflow phases, the exact 26 Management API
read scopes, the
data.report.summary[]parsing, the full finding→plan triage table(including the "Custom Domains is Free, not an upgrade" caveat), the mandatory fusion
checklist +
greplint, and the two-loop (Immediate / After-Upgrading) gated remediation.Restored coverage & fixes in this PR
tests/evals.json+graders.jsonareported to
evals/behavioral/cases/audit.json, matching the consolidated harness atrepo-root
evals/(the per-skilltests/layout was retired in the v2.0 migration).feature-audit-remediation.md: pick theauth0 apimethod from the path column(
patchfor most resources,putfor MFA/prompt toggles,postfor creates —patchis not universal), and note that
auth0 apidefaults toGETwithout--data/POSTwith
--data, so the method must always be stated explicitly.Gates (all green)
check_router_reachability.py— all references routable, no second-hop linkscheck_routing_evals.py— routing cases resolve, incl. the newauditcaseuvx skillsaw --strict— 0 errors / 0 warnings, Grade A🤖 Generated with Claude Code
Summary by CodeRabbit
audit-tenantrouting case.