You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,12 @@
1
1
# Changelog
2
2
3
+
## [2.0.0] - 2026-09-01
4
+
### Changed : chaîne de déploiement et de preuve v2
5
+
-**Rupture de compatibilité : manifests v2 et portée organisationnelle stricte.** Les artefacts de déploiement portent désormais leur contrat de version, l'organisation cible est vérifiée sans repli implicite, et un garde-fou bloque les collisions avant toute écriture.
6
+
-**Audit visuel fail-closed et Playwright durci.** Une preuve visuelle manquante ou invalide fait échouer l'audit ; l'exécution du navigateur applique les limites et contrôles de sécurité attendus.
7
+
-**Release reproductible avec SBOM.** La chaîne de publication produit et vérifie ses artefacts de supply chain, tandis que la CI couvre Grafana 9 à 13.
8
+
-**Registre de prix rafraîchi depuis les sources officielles.** Chaque entrée expose sa provenance et sa date de vérification ; les tarifs absents restent non chiffrés. Artificial Analysis n'est disponible qu'en fallback tiers opt-in, avec attribution et restrictions de redistribution explicites.
9
+
3
10
## [1.6.0] - 2026-08-30
4
11
### Fixed : exploitabilité en production
5
12
-**Un 403 de permissions sortait une trace Python.** Un exploitant ne pouvait pas savoir qu'il s'agissait d'un rôle de jeton insuffisant, ni dans quel état il avait laissé l'instance. Message actionnable nommant le rôle requis, codes de sortie distincts (3 = rien écrit, 4 = déploiement partiel), et l'emplacement des JSON générés pour un import manuel.
Copy file name to clipboardExpand all lines: README.md
+27-6Lines changed: 27 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -110,6 +110,24 @@ The skill handles discovery → registry refresh → blueprint selection → dep
110
110
<summary><b>As a standalone CLI (no AI required)</b></summary>
111
111
112
112
Pure Python 3.8+ stdlib. No pip install. The three commands at the top of this README are the whole workflow. `--dry-run` writes JSON without touching your instance; `--selftest` renders all seven blueprints offline from a simulated capability map.
113
+
114
+
Unknown or officially unpriced detected models can use an explicit third-party
It is off by default. Only the fixed Artificial Analysis Free endpoint is
124
+
called, only for detected models that need a price, and only the
125
+
`ARTIFICIAL_ANALYSIS_API_KEY` environment variable is accepted. The key is not
126
+
logged or written. Results are median multi-provider estimates, marked and
127
+
attributed in dashboards and recording-rule labels, with a 24-hour atomic
128
+
overlay in `model_registry.artificial-analysis.cache.json` next to the
129
+
capability map. It never contains the merged registry. Official pricing always
130
+
wins. See [pricing provenance](references/pricing_provenance.md).
113
131
</details>
114
132
115
133
<details>
@@ -169,10 +187,13 @@ Or delete it from the UI: Dashboards → AI Observability → Folder settings
169
187
Delete. Generated recording rules are separate: they are a file you copied into
170
188
Prometheus, so removing them is removing that file and reloading.
171
189
172
-
**What leaves your network: nothing.** The scripts talk only to your Grafana,
173
-
which talks to your own Prometheus, Loki and Tempo. The one optional exception is
174
-
the model-price refresh, a web search you can skip entirely; the bundled
175
-
registry works offline and the dashboards display its verification date.
190
+
**What leaves your network: nothing by default.** The scripts talk only to your
191
+
Grafana, which talks to your own Prometheus, Loki and Tempo. The explicit
192
+
`--pricing-fallback artificial-analysis` exception sends authenticated GET
193
+
requests to `https://artificialanalysis.ai/api/v2/language/models/free`; it can
194
+
be skipped entirely. No Grafana data, including detected model names, is sent
195
+
in the request: model matching happens locally against the unfiltered paginated
196
+
catalog. The bundled registry works offline.
176
197
177
198
**What an agent sees.** Used as a skill, Claude reads the capability map, which
178
199
contains metric names, model names, and team or service label values from your
@@ -186,7 +207,7 @@ Skills execute code, and [a 2026 Snyk audit found 36% of published skills had at
186
207
187
208
-**Zero dependencies.** Python stdlib only (`urllib`, `json`, `hashlib`). ~2,000 lines total. Playwright is *optional*, only for the visual-audit fallback.
188
209
-**Least privilege.** Works with an Editor service-account token. Alert provisioning degrades gracefully on 403 (exports JSON for manual import).
189
-
-**No secret leakage.**The token is never logged, never embedded in dashboards, never placed in URLs.
210
+
-**No secret leakage.**Grafana credentials and the optional Artificial Analysis key are never logged, embedded in dashboards, persisted, or placed in URLs.
190
211
-**No prompt-content capture.**`gen_ai.input/output.messages` stay off by default; the docs treat enabling them as a GDPR decision, not a flag.
191
212
-**Idempotent & reversible.** Deterministic UIDs, one folder, `overwrite` semantics: delete the folder and it's gone.
192
213
-**Offline-testable.**`--selftest` + `tests/audit_harness.py` (4 simulated instance topologies plus regression tests) runs with zero network, `tests/live_query_check.py` executes every generated query against a real Prometheus, and the demo stack gives a full end-to-end deploy. That's the CI; the badge above is the live workflow status.
@@ -231,7 +252,7 @@ tests/audit_harness.py # offline checks across 4 instance topologies + re
231
252
232
253
**Grafana Cloud?** Yes. Cloud is auto-detected; the image renderer is built in, so visual audit works out of the box.
233
254
234
-
**My models aren't in the registry.** They appear in an "unpriced models" panel instead of being billed wrong. Add a price or alias to `model_registry.json`, re-forge. (The matcher scores by specificity: `gpt-5.4-mini` will never be billed at `gpt-5.4` rates. There is a test for that.)
255
+
**My models aren't in the registry.** They appear in an "unpriced models" panel instead of being billed wrong. Add a price or alias to `model_registry.json`, or explicitly opt in to `--pricing-fallback artificial-analysis`. The fallback accepts only a unique exact normalized ID, slug, name, or alias match. Ambiguous, absent, null, or failed responses stay unpriced.
235
256
236
257
**Can I publish the generated dashboards?** Yes: `--export-portable` emits JSON with `__inputs`/`${DS_PROMETHEUS}` placeholders, the format grafana.com/dashboards requires.
Copy file name to clipboardExpand all lines: SKILL.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -47,6 +47,20 @@ Produces the capability map: version/edition/namespace, API availability (legacy
47
47
48
48
Read `references/model_registry.json`. If `_meta.verified_at` is more than 30 days old AND web search is available: refresh the prices of the models actually present in the capability map (not the whole registry) from the URLs in `_meta.sources`, then write `model_registry.local.json` next to the capability map. The generator loads the local file first. Without web access, use the seed as-is; cost dashboards display the registry date in their description.
49
49
50
+
Artificial Analysis is a third-party fallback, never an automatic refresh.
51
+
Use `--pricing-fallback artificial-analysis` only after the user opts in and
52
+
only with `ARTIFICIAL_ANALYSIS_API_KEY` in the environment. The forge calls the
53
+
fixed Free endpoint, caches results next to the capability map for 24 hours,
54
+
and accepts only a unique exact normalized ID, slug, name, or alias match.
55
+
The cache is the separate overlay
56
+
`model_registry.artificial-analysis.cache.json`; it never replaces the official
57
+
local registry `model_registry.local.json` or the seed.
58
+
Official prices always win. Ambiguous, absent, null, invalid, unauthorized,
59
+
rate-limited, or server-error responses leave the model unpriced and do not
60
+
stop the forge. Relay that these values are Artificial Analysis median
61
+
multi-provider estimates. Never print or persist the key. Full protocol:
Every run writes `prometheus_rules_llmops.yml`: prices become series (`llm:price_*_usd_per_token{<model_label>=…}`) and cost becomes an aggregate metric (`llm:cost_usd_per_second`) joined by vector matching. Two files are written: the portable rule file and a `PrometheusRule` manifest for Kubernetes clusters running the Prometheus Operator, which is where most enterprise deployments live. Same rules, different packaging; the loading recipe per backend (Mimir, Thanos, VictoriaMetrics, AMP, Google Managed Prometheus) is in `references/grafana_api_compat.md` §6. Copy the portable file into Prometheus `rule_files` and reload: on the next run `discover.py` detects the `recorded` dialect and cost panels drop from a 2N-term sum to an O(1) query: unlimited models, and prices updatable without regenerating dashboards. Without it, on-the-fly composition stays active (40-model ceiling).
Copy file name to clipboardExpand all lines: references/locale.fr.json
+3-1Lines changed: 3 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -89,6 +89,8 @@
89
89
"The low tail is the real signal; the mean hides the failures.": "La queue basse est le vrai signal : la moyenne masque les échecs.",
90
90
"The shape of the mix tells you the real strategy: a cheaper model gaining share means a FinOps trade-off that is working.": "La forme du mix raconte la stratégie réelle : montée d'un modèle économique = arbitrage FinOps qui fonctionne.",
91
91
"These models are observed but **excluded from cost** (unknown price):": "Ces modèles sont observés mais **exclus du calcul de coût** (prix inconnu) :",
0 commit comments