v2.0.2

What's Changed

• fix: harden Grafana Cloud host detection by @alebgl77 in #2
• chore: prepare v2.0.2 release by @alebgl77 in #3

New Contributors

• @alebgl77 made their first contribution in #2

Full Changelog: v2.0.1...v2.0.2

v2.0.1

Full Changelog: v2.0.0...v2.0.1

v2.0.0

Grafana LLMOps Forge v2.0.0

Production-readiness release focused on safe multi-organization deployment, verifiable artifacts, visual evidence, and honest model-cost provenance.

Breaking changes

• Deployment manifests now use the v2 contract.
• Grafana organization scope is resolved and verified without an implicit fallback.
• Dashboard and alert identity collisions fail before any write.

Highlights

• Visual audits now fail closed when evidence is missing or invalid, with hardened Playwright credential and origin handling.
• Release artifacts are reproducible and include a deterministic SPDX 2.3 SBOM, SHA-256 checksums, SLSA provenance, and an SPDX attestation.
• CI covers Python 3.9 and 3.13, Windows portability, live Prometheus queries, and Grafana 9 through 13.
• The model registry was refreshed from official provider sources and now records price provenance and verification dates.
• Models without a reliable official price remain unpriced instead of receiving an invented value.
• Artificial Analysis is available only as an explicit fallback for missing prices. Values are marked as third-party estimates, attributed to Artificial Analysis, cached locally, and never written into the public seed registry.

Verification

• Signed commit: 69846d2a01f70f2bf7bd64f8f8a47c87047639a6
• Signed tag: v2.0.0
• CI and release workflows completed successfully.
• Release publication occurs only after both artifact attestations succeed.

Full comparison: v1.6.0...v2.0.0

v1.6.0 — English by default, three governance frameworks, production-ready failure behaviour

Everything since v1.2.3, driven by repeated adversarial audits. Every defect
fixed here was silent — nothing crashed, the dashboards rendered, the numbers
were wrong.

The product now speaks to its audience

• Dashboards, alerts and rules render in English by default. They were French:
  a platform team outside France deployed this and read Dépense par souveraineté
  on its screens. --locale fr keeps the French labels; adding a language is a
  JSON file, not a code change.
• Governance reads against the EU AI Act, ISO/IEC 42001 and NIST AI RMF
  (--framework). The same log volume evidences Art. 12, A.6.2.8 and MANAGE 4.1,
  so the measured panels are identical — only the reading changes. The board also
  states what it does not prove.
• Rules ship in two formats: the portable file (Prometheus, Thanos, Mimir,
  VictoriaMetrics, AMP, Grafana Cloud) and a PrometheusRule manifest for
  Kubernetes under the Prometheus Operator. --rules-window / --rules-interval
  adapt to your scrape interval.

Silent defects fixed

• output-token cost vanished from the recording rules (or dropping half the
  join) — undercounting by ~6x with no error shown
• recording rules produced nothing at all: prices and cost sat in concurrently
  evaluated groups
• PromQL escaping broken across the string layer: the AI Act sovereignty panel
  failed for every model name with a hyphen or a dot
• exporter namespace prefixes and UTF-8 metric names made every signal invisible
• datasource timeInterval unset, silently skewing every rate()

Production readiness

• a permissions 403 now yields an actionable message and a distinct exit code
  instead of a traceback; a refused dashboard no longer aborts the rest
• the client has no delete path — the blast-radius guarantee is structural,
  and reversal is one folder removal, documented
• supply chain: actions pinned to commit SHAs, read-only token by default, no
  expression interpolation in run blocks, zizmor in CI, the .skill built
  reproducibly by CI and never committed
• NVIDIA SkillSpector: zero findings on the shipped skill

Verification

161 offline checks across simulated topologies, every generated query executed
against a real Prometheus, value invariants proving the numbers agree with each
other, a full end-to-end deploy on the demo stack, and fault injection against a
simulated Grafana — all in CI.

v1.2.1 — PromQL escaping fix, live query test

Fixed

PromQL regex escaping across the string layer — found by the new live check, invisible to offline validation. re.escape emits a backslash-hyphen, which RE2 rejects; and a single backslash-dot is consumed by the matcher's quoted string before reaching the regex engine. The governance sovereignty panel was broken for every model name containing a hyphen or a dot — i.e. nearly all of them.

Added

• tests/live_query_check.py — probes a real Prometheus, builds the capability map from what it finds, forges, and executes every generated expression. Wired as a CI job.
• Documentation-drift guard: the CI badge is now the real GitHub Actions status, and the harness fails if the README carries a stale hardcoded test count.

Hardened

• .gitignore rule order (*.skill was silently re-ignoring dist/*.skill).
• CI matrix moved off Python 3.8 (EOL, unstable on ubuntu-24.04); the image-renderer plugin install is now optional so the e2e job does not depend on it at boot.

Verified on real data

63/63 generated expressions return data on a live Prometheus, in inline and recorded mode — cost auto-switches to sum(llm:cost_usd_per_second) once the generated recording rules are loaded.

v1.2.0 — scalable cost, burn-rate SLOs, real demo stack

Fixed (5 real bugs, each with a regression test)

• The signal-lost alert stayed silent when the signal was actually lost — noDataState: OK muted the rule if the datasource became unreachable. Now Alerting.
• Governance Loki panel typed as prometheus (also broke portable export).
• Hardcoded orgID: 1 in alert rules → real org read from /api/org.
• $__rate_interval inside alert rules → explicit windows.
• Silent multi-datasource pick → flagged by discovery, resolved with --datasource.

Added

• Cost recording rules: prices become series, cost becomes llm:cost_usd_per_second joined by vector matching. FinOps panels go O(1), unlimited models, prices updatable without regenerating dashboards. --cost-mode auto|recorded|inline.
• Multi-window burn-rate alerting (5m/1h page, 30m/6h ticket) on the SLO error budget, --slo-target.
• 7th blueprint — Quality & Evals (RAGAS / LLM-judge scores, guardrails), signal-gated.
• Exemplars + metric→trace links to Tempo, with datasource exemplar-routing detection.
• --export-portable — __inputs/${DS_*} JSON, the format grafana.com/dashboards requires.
• make demo — Grafana + Prometheus + a synthetic LLM metrics emitter, also wired as a real end-to-end CI job.

Hardened

• Cardinality guardrail (no group-by above 300 distinct values) and topk bounds on every grouped panel; maxDataPoints on all timeseries.
• SKILL.md fully in English (French version in docs/SKILL.fr.md).
• Audit harness 27 → 45 checks.

Install: download dist/grafana-llmops-forge.skill → Claude → Save skill. Or make demo and watch it work.

v1.1.0 — vision-verified forge

First public release.

Install as a skill: download dist/grafana-llmops-forge.skill → Claude → Save skill.
Or plain CLI (zero deps): see README — 3 commands.

Highlights: 6 blueprints · 4 telemetry dialects · 30-model sovereignty-aware price registry · 5 SLO alerts · visual verification loop · 27-check offline CI.

v0.1.0 — first public release

First public release. See CHANGELOG.md. Try it offline in seconds: python3 scripts/forge_dashboards.py --selftest.