release: v0.2.0 — MLFLOW_FALSIFY_TAG_SCOPE + tag_experiment helper

Surfaced by @smqd19 on mlflow/mlflow#23369: HPO sweeps at 50k runs/day
repeat 5 identical descriptive PRML tags per-run. Wasteful.

This release adds MLFLOW_FALSIFY_TAG_SCOPE=run|experiment. The
audit-essential commitment (prml.manifest_hash, prml.manifest_path)
stays per-run. The 5 descriptive fields (metric, comparator, threshold,
dataset_id, version) lift to experiment level via the new
mlflow_falsify.tag_experiment() helper.

Backward compatible: default scope is "run", emitting all 7 tags
per-run exactly as v0.1.x did.

Closes #1.

Author: Cüneyt Öztürk

CHANGELOG.md

@@ -0,0 +1,31 @@
+# Changelog
+
+All notable changes to mlflow-falsify will be documented in this file. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.2.0] - 2026-05-23
+
+### Added
+- `MLFLOW_FALSIFY_TAG_SCOPE` environment variable. When set to `experiment`, only `prml.manifest_hash` and `prml.manifest_path` attach per-run; the 5 descriptive tags (metric, comparator, threshold, dataset_id, version) are lifted off-run.
+- `mlflow_falsify.tag_experiment(experiment=None, *, manifest_path=None)` helper. Sets the descriptive PRML tags at experiment level via `MlflowClient.set_experiment_tag`. Idempotent.
+- README section: "HPO sweeps and tag scope".
+- 4 new unit tests covering default scope, experiment scope, unknown-value fallback, and explicit-path computation.
+
+### Why
+Surfaced by [@smqd19](https://github.com/smqd19) on [mlflow/mlflow#23369](https://github.com/mlflow/mlflow/discussions/23369): at 50k HPO runs/day, emitting 5 identical descriptive tags per-run is wasteful. The audit-essential commitment (`prml.manifest_hash`) must stay per-run; the descriptive fields can live at experiment scope. Tracks [#1](https://github.com/studio-11-co/mlflow-falsify/issues/1).
+
+### Backward compat
+Default behaviour unchanged. Without setting the env var, all 7 tags attach per-run exactly as in v0.1.x.
+
+## [0.1.3] - 2026-05-20
+- README: switch DOI badge to shields.io format for reliable rendering.
+- README: add Audit & compliance crosswalks section (EU AI Act Art. 12, NIST AI RMF, ISO/IEC 42001).
+
+## [0.1.2] - 2026-05-16
+- CI: defer `FalsifyRunContextProvider` import to break MLflow circular load during entry-point discovery.
+- Add OpenSSF Scorecard workflow.
+
+## [0.1.1] - 2026-05-15
+- First public release on PyPI.
+- 5 unittests including TV-001 conformance assertion against the PRML v0.1 normative vector.
+- Discovery via standard `mlflow.run_context_provider` entry point.
+- Canonicalize logic vendored byte-for-byte from `falsify` v0.1.4.

README.md

@@ -41,6 +41,27 @@ When a `.prml.yaml` or `prml.yaml` is found in the current directory or any ance
 
 Missing or malformed fields are silently skipped. The provider never raises into your run.
 
+## HPO sweeps and tag scope
+
+In an HPO sweep the same PRML claim repeats across thousands of runs, so emitting the 5 descriptive tags per-run becomes pure tag noise. As of v0.2.0 the plugin supports lifting them to experiment level:
+
+```bash
+export MLFLOW_FALSIFY_TAG_SCOPE=experiment
+```
+
+```python
+import mlflow_falsify
+
+mlflow.set_experiment("credit-scorer-hpo")
+mlflow_falsify.tag_experiment()  # idempotent; sets metric/comparator/threshold/dataset_id/version once
+
+for params in hpo_grid:
+    with mlflow.start_run():
+        ...  # only prml.manifest_hash and prml.manifest_path attach per-run
+```
+
+Default behaviour (`MLFLOW_FALSIFY_TAG_SCOPE=run` or unset) is unchanged: all 7 tags attach per-run, backward-compatible with v0.1.x.
+
 ## Why this matters
 
 - **EU AI Act Article 12 evidence layer.** Every logged run carries a tamper-evident pointer to the claim it was meant to test.

mlflow_falsify/__init__.py

@@ -10,12 +10,93 @@
 
 from __future__ import annotations
 
-__version__ = "0.1.3"
-__all__ = ["FalsifyRunContextProvider"]
+from pathlib import Path
+from typing import Dict, Optional, Union
+
+__version__ = "0.2.0"
+__all__ = ["FalsifyRunContextProvider", "tag_experiment"]
 
 
 def __getattr__(name: str):
     if name == "FalsifyRunContextProvider":
         from mlflow_falsify.run_context import FalsifyRunContextProvider
         return FalsifyRunContextProvider
+    if name == "tag_experiment":
+        return tag_experiment
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def tag_experiment(
+    experiment: Optional[Union[str, int]] = None,
+    *,
+    manifest_path: Optional[Union[str, Path]] = None,
+) -> Dict[str, str]:
+    """Lift the descriptive PRML tags to experiment-level once per sweep.
+
+    For HPO sweeps where a single PRML claim is reused across thousands of
+    runs, repeating ``prml.metric`` / ``prml.comparator`` / ``prml.threshold``
+    / ``prml.dataset_id`` / ``prml.version`` per-run is wasteful. Setting
+    ``MLFLOW_FALSIFY_TAG_SCOPE=experiment`` keeps only ``prml.manifest_hash``
+    and ``prml.manifest_path`` at the run level; this helper sets the rest
+    once at experiment level via ``MlflowClient.set_experiment_tag``.
+
+    Idempotent: calling twice with the same manifest is a no-op.
+
+    Parameters
+    ----------
+    experiment :
+        Experiment name (``str``) or experiment id (``int`` or numeric
+        ``str``). If omitted, the currently active MLflow experiment is
+        used.
+    manifest_path :
+        Optional explicit path to a ``.prml.yaml`` manifest. If omitted,
+        the manifest is auto-discovered from CWD upwards (the same rule
+        the provider uses).
+
+    Returns
+    -------
+    Dict[str, str]
+        The tags that were written. Empty dict if no manifest was found
+        or the manifest contained no descriptive fields.
+
+    Raises
+    ------
+    RuntimeError
+        If ``mlflow`` is not installed.
+    ValueError
+        If the named experiment cannot be resolved.
+    """
+    try:
+        import mlflow
+        from mlflow.tracking import MlflowClient
+    except ImportError as exc:  # pragma: no cover
+        raise RuntimeError(
+            "mlflow_falsify.tag_experiment requires mlflow to be installed"
+        ) from exc
+
+    from mlflow_falsify.run_context import _RUN_LEVEL_ALWAYS, _compute_tags
+
+    path = Path(manifest_path) if manifest_path is not None else None
+    all_tags = _compute_tags(path)
+    descriptive = {k: v for k, v in all_tags.items() if k not in _RUN_LEVEL_ALWAYS}
+    if not descriptive:
+        return {}
+
+    client = MlflowClient()
+
+    if experiment is None:
+        exp_id = mlflow.tracking.fluent._get_experiment_id()
+    elif isinstance(experiment, int) or (
+        isinstance(experiment, str) and experiment.isdigit()
+    ):
+        exp_id = str(experiment)
+    else:
+        exp = client.get_experiment_by_name(str(experiment))
+        if exp is None:
+            raise ValueError(f"experiment {experiment!r} not found")
+        exp_id = exp.experiment_id
+
+    for key, value in descriptive.items():
+        client.set_experiment_tag(exp_id, key, value)
+
+    return descriptive

mlflow_falsify/run_context.py

@@ -36,6 +36,20 @@ def tags(self) -> Dict[str, str]:
 
 MANIFEST_FILENAMES = (".prml.yaml", "prml.yaml")
 
+# Tag-scope routing. Two tags are audit-essential and always emitted per-run:
+#   prml.manifest_hash  — the SHA-256 commitment this run is bound to
+#   prml.manifest_path  — the file the provider resolved (for diagnostics)
+# The remaining descriptive fields can optionally be lifted to experiment
+# level via `mlflow_falsify.tag_experiment()` in HPO sweeps where the
+# claim is identical across thousands of runs. Controlled by env var
+# MLFLOW_FALSIFY_TAG_SCOPE: "run" (default) or "experiment".
+_RUN_LEVEL_ALWAYS = ("prml.manifest_hash", "prml.manifest_path")
+
+
+def _tag_scope() -> str:
+    raw = os.environ.get("MLFLOW_FALSIFY_TAG_SCOPE", "run").strip().lower()
+    return raw if raw in ("run", "experiment") else "run"
+
 
 def _find_manifest(start: Optional[Path] = None) -> Optional[Path]:
     """Walk up from `start` (default CWD) looking for a PRML manifest."""
@@ -85,45 +99,56 @@ def in_context(self) -> bool:
         return _find_manifest() is not None
 
     def tags(self) -> Dict[str, str]:
-        result: Dict[str, str] = {}
+        result = _compute_tags()
+        if _tag_scope() == "experiment":
+            return {k: v for k, v in result.items() if k in _RUN_LEVEL_ALWAYS}
+        return result
+
+
+def _compute_tags(manifest_path: Optional[Path] = None) -> Dict[str, str]:
+    """Compute the full PRML tag set for the manifest at `manifest_path`
+    (or auto-discovered from CWD if None). Returns an empty dict when no
+    manifest is found or parseable. Never raises."""
+    result: Dict[str, str] = {}
+    if manifest_path is None:
         manifest_path = _find_manifest()
-        if manifest_path is None:
-            return result
+    if manifest_path is None:
+        return result
 
-        result["prml.manifest_path"] = _relative_path(manifest_path)
+    result["prml.manifest_path"] = _relative_path(manifest_path)
 
-        spec = _load_manifest(manifest_path)
-        if spec is None:
-            return result
+    spec = _load_manifest(manifest_path)
+    if spec is None:
+        return result
 
-        # Hash the canonical bytes. Anything raised here is swallowed —
-        # tag emission must never break the run.
-        try:
-            result["prml.manifest_hash"] = manifest_hash(spec)
-        except Exception:
-            pass
+    # Hash the canonical bytes. Anything raised here is swallowed —
+    # tag emission must never break the run.
+    try:
+        result["prml.manifest_hash"] = manifest_hash(spec)
+    except Exception:
+        pass
 
-        version = _safe_str(spec.get("version"))
-        if version is not None:
-            result["prml.version"] = version
+    version = _safe_str(spec.get("version"))
+    if version is not None:
+        result["prml.version"] = version
 
-        metric = _safe_str(spec.get("metric"))
-        if metric is not None:
-            result["prml.metric"] = metric
+    metric = _safe_str(spec.get("metric"))
+    if metric is not None:
+        result["prml.metric"] = metric
 
-        comparator = _safe_str(spec.get("comparator"))
-        if comparator is not None:
-            result["prml.comparator"] = comparator
+    comparator = _safe_str(spec.get("comparator"))
+    if comparator is not None:
+        result["prml.comparator"] = comparator
 
-        threshold = spec.get("threshold")
-        threshold_str = _safe_str(threshold)
-        if threshold_str is not None:
-            result["prml.threshold"] = threshold_str
+    threshold = spec.get("threshold")
+    threshold_str = _safe_str(threshold)
+    if threshold_str is not None:
+        result["prml.threshold"] = threshold_str
 
-        dataset = spec.get("dataset")
-        if isinstance(dataset, dict):
-            dataset_id = _safe_str(dataset.get("id"))
-            if dataset_id is not None:
-                result["prml.dataset_id"] = dataset_id
+    dataset = spec.get("dataset")
+    if isinstance(dataset, dict):
+        dataset_id = _safe_str(dataset.get("id"))
+        if dataset_id is not None:
+            result["prml.dataset_id"] = dataset_id
 
-        return result
+    return result

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mlflow-falsify"
-version = "0.1.3"
+version = "0.2.0"
 description = "MLflow plugin: automatic PRML manifest hash tagging for runs. Pre-registered ML evaluation claims."
 readme = "README.md"
 requires-python = ">=3.9"

tests/test_run_context.py

@@ -112,5 +112,79 @@ def test_malformed_manifest_does_not_raise(self) -> None:
                 self.assertNotIn("prml.manifest_hash", tags)
 
 
+class TagScopeTests(unittest.TestCase):
+    """MLFLOW_FALSIFY_TAG_SCOPE env var routes descriptive tags off-run."""
+
+    def _with_scope(self, value: str | None):
+        # Context manager helper: set env var, restore on exit.
+        class _Scope:
+            def __init__(self, v: str | None) -> None:
+                self.v = v
+                self.prev: str | None = None
+                self.had: bool = False
+
+            def __enter__(self_inner) -> None:
+                self_inner.had = "MLFLOW_FALSIFY_TAG_SCOPE" in os.environ
+                self_inner.prev = os.environ.get("MLFLOW_FALSIFY_TAG_SCOPE")
+                if self_inner.v is None:
+                    os.environ.pop("MLFLOW_FALSIFY_TAG_SCOPE", None)
+                else:
+                    os.environ["MLFLOW_FALSIFY_TAG_SCOPE"] = self_inner.v
+
+            def __exit__(self_inner, *exc: object) -> None:
+                if self_inner.had:
+                    os.environ["MLFLOW_FALSIFY_TAG_SCOPE"] = self_inner.prev or ""
+                else:
+                    os.environ.pop("MLFLOW_FALSIFY_TAG_SCOPE", None)
+
+        return _Scope(value)
+
+    def test_default_scope_emits_all_tags(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp).resolve()
+            (root / ".prml.yaml").write_text(TV_001_YAML, encoding="utf-8")
+            with _Chdir(root), self._with_scope(None):
+                tags = FalsifyRunContextProvider().tags()
+            self.assertEqual(tags.get("prml.manifest_hash"), TV_001_HASH)
+            self.assertEqual(tags.get("prml.metric"), "accuracy")
+            self.assertEqual(tags.get("prml.threshold"), "0.85")
+            self.assertEqual(tags.get("prml.dataset_id"), "imagenet-val-2012")
+
+    def test_experiment_scope_keeps_only_audit_essential(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp).resolve()
+            (root / ".prml.yaml").write_text(TV_001_YAML, encoding="utf-8")
+            with _Chdir(root), self._with_scope("experiment"):
+                tags = FalsifyRunContextProvider().tags()
+            # Audit-essential tags stay
+            self.assertEqual(tags.get("prml.manifest_hash"), TV_001_HASH)
+            self.assertIn("prml.manifest_path", tags)
+            # Descriptive tags lifted off-run
+            self.assertNotIn("prml.metric", tags)
+            self.assertNotIn("prml.comparator", tags)
+            self.assertNotIn("prml.threshold", tags)
+            self.assertNotIn("prml.dataset_id", tags)
+            self.assertNotIn("prml.version", tags)
+
+    def test_unknown_scope_falls_back_to_run(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp).resolve()
+            (root / ".prml.yaml").write_text(TV_001_YAML, encoding="utf-8")
+            with _Chdir(root), self._with_scope("nonsense"):
+                tags = FalsifyRunContextProvider().tags()
+            self.assertEqual(tags.get("prml.metric"), "accuracy")
+
+    def test_compute_tags_with_explicit_path(self) -> None:
+        from mlflow_falsify.run_context import _compute_tags
+
+        with tempfile.TemporaryDirectory() as tmp:
+            root = Path(tmp).resolve()
+            manifest = root / "subdir" / "claim.yaml"
+            manifest.parent.mkdir()
+            manifest.write_text(TV_001_YAML, encoding="utf-8")
+            tags = _compute_tags(manifest)
+            self.assertEqual(tags.get("prml.manifest_hash"), TV_001_HASH)
+
+
 if __name__ == "__main__":
     unittest.main()