feat: add safe Direct UTM and keyword bid workflows

Author: Адель

AGENTS.md

@@ -30,6 +30,64 @@ Purpose: quick onboarding for external humans and generic agents (including codi
 - Marketing / semantic tasks: use `docs/MARKETER_GUIDE.md` and corresponding DirectPilot endpoints.
 - If a direct Yandex API method exists in DirectPilot, **do not** call raw Yandex endpoints (`api.direct.yandex.com`, `api-metrika.yandex.net`, AI Studio/Search API) directly.
 
+## Human Approval Contract (mandatory for all write operations)
+
+Technical `approved=true` in a request body is NOT a self-approval by an agent.
+For any **real Direct/Metrika write** (minuses, keywords, ads, moderation,
+strategy, bids, budgets, time-targeting, autotargeting, goals, UTM, pause/resume,
+live-create, bids modifiers, etc.) the agent/marketer/script MUST follow this
+sequence:
+
+1. **Read-only / dry-run preview** — gather current state via read endpoints and
+   `dry_run=true` previews. Produce a concrete diff/impact summary (what entities,
+   what will change, expected effect, risk).
+2. **Show to user** — present the preview and diff in chat, ticket, or operator
+   UI. Never skip this step.
+3. **Get explicit user confirmation** — the user must explicitly approve the
+   action in the conversation/ticket/UI. The agent must not interpret silence,
+   an unrelated "ok", or a general instruction as approval for a specific write.
+4. **Apply only after confirmation** — only then send the write with
+   `approved=true`, `idempotency_key`, `dry_run=false`, and
+   `DIRECTPILOT_MODE=live_write`.
+5. **Readback** — verify what changed using the operation readback in the
+   response or a dedicated follow-up read endpoint.
+
+Marketers do NOT apply live writes themselves. They prepare recommendations and
+read-only/dry-run outputs only. The apply step is performed by an operator or
+orchestrator after explicit user approval.
+
+## UTM workflow for external agents (summary)
+
+UTM is the primary mechanism for tracking ad performance in Yandex Metrika.
+Every external agent (marketing, coding, reviewer) must follow this workflow:
+
+**Existing campaigns:**
+1. `GET /yandex/campaigns/{campaign_id}/utm-audit` — read-only audit
+2. `POST /yandex/campaigns/{campaign_id}/utm-plan` — always dry_run, preview only; preserves query/fragment and emits concrete old_url → new_url
+3. **User approval** (see Human Approval Contract above) — confirm slug, overwrite, sitelinks
+4. `POST /yandex/campaigns/{campaign_id}/utm-apply` — with gates
+5. Readback — verify confirmed URLs from response
+
+**New campaigns (live-create):**
+- Include `utm_config` in the `POST /yandex/campaigns/live-create` request to
+  tag ads with UTM at birth.
+- `LiveCreateCampaignRequest.utm_config` overrides `draft.utm_config` when both
+  are set — the request-level config takes precedence.
+- If `utm_config` is omitted, the draft's stored `utm_config` is used as fallback.
+
+**When to ask the user:**
+- Campaign slug/naming is unclear.
+- **Cyrillic campaign names:** when the campaign name contains only Cyrillic
+  characters, the automatic slug generator falls back to ``campaign-{id}``
+  because transliteration drops non-ASCII chars.  For readable Metrika reports,
+  ask the user to provide an explicit ``campaign_slug`` (Latin transliteration
+  or semantic slug, e.g. ``turbiny-rostov`` instead of ``campaign-12345``).
+- Overwrite of existing UTM is requested.
+- Sitelinks UTM apply is desired; `sitelink` part is preview-only and apply is currently `not_implemented` (fail-closed).
+
+See `docs/MARKETER_GUIDE.md` (UTM section) and `docs/API_SIMPLE.md`
+for the full endpoint contracts.
+
 ## Public skill vs private local skills
 - This repository is self-contained for external humans and agents. Start with `AGENTS.md`, then read `skills/directpilot-operations/SKILL.md` and the docs listed above.
 - Private Hermes skills such as a maintainer's local `yandex-direct-api` skill are convenience memory for that maintainer only. They are not required to operate or contribute to this repository.

app/main.py

@@ -96,6 +96,17 @@
     YandexAutotargetingReadResult,
     YandexAutotargetingRequest,
     YandexAutotargetingResult,
+    # UTM
+    UtmAuditResult,
+    UtmApplyRequest,
+    UtmApplyResult,
+    UtmConfig,
+    UtmPlanRequest,
+    UtmPlanResult,
+    # Keyword bids
+    KeywordBidItem,
+    KeywordBidUpdateRequest,
+    KeywordBidUpdateResult,
 )
 from app.store import store
 from app.yandex_direct import YandexDirectClient, YandexDirectError
@@ -3133,6 +3144,212 @@ def yandex_autotargeting_update(
         ) from exc
 
 
+# ---------------------------------------------------------------------------
+# Keyword bids update — live-safe SearchBid / ContextBid changes
+# ---------------------------------------------------------------------------
+
+
+@app.post(
+    "/yandex/campaigns/{campaign_id}/bids",
+    response_model=KeywordBidUpdateResult,
+    responses={
+        409: {
+            "description": "Safety gate or idempotency conflict: missing approval, non-live_write apply, or replay payload mismatch.",
+        },
+        502: {
+            "description": "Upstream Yandex Direct keywordbids.set / readback failure, with redacted diagnostics only.",
+        },
+    },
+)
+def yandex_keyword_bids_update(
+    campaign_id: str,
+    payload: KeywordBidUpdateRequest,
+    settings: Settings = Depends(get_settings),
+    client: YandexDirectClient | None = Depends(get_yandex_client),
+) -> KeywordBidUpdateResult:
+    """Update SearchBid / ContextBid for existing keywords via v5 ``keywordbids.set``.
+
+    Standard product gate contract:
+    ``dry_run=True`` (default) is preview-only and never performs a network
+    write; the response includes the exact v5 ``keywordbids.set`` payload
+    that WOULD be sent, with ``applied=False``.
+
+    ``dry_run=False`` requires ``DIRECTPILOT_MODE=live_write``,
+    ``approved=True`` and a valid ``idempotency_key``.
+
+    Request items use RUBLES at the REST boundary; the store converts to
+    Direct micros (× 1 000 000). The minimal v5 item shape is
+    ``KeywordId + SearchBid`` / ``KeywordId + ContextBid`` — no
+    CampaignId / AdGroupId in the item.
+
+    After apply, the endpoint reads back keyword bids for the campaign
+    and returns the changed keyword ids with current Bid/ContextBid.
+    """
+    if not payload.approved:
+        raise HTTPException(
+            status_code=409,
+            detail="Action requires explicit approval before keyword bids update",
+        )
+    if not payload.dry_run and settings.directpilot_mode != "live_write":
+        raise HTTPException(
+            status_code=409,
+            detail=(
+                f"Live writes require DIRECTPILOT_MODE=live_write; "
+                f"current mode is {settings.directpilot_mode!r}; "
+                f"keyword bids apply is not allowed in this mode "
+                f"(dry_run=True is the only allowed path)"
+            ),
+        )
+    try:
+        return store.yandex_keyword_bids_update(
+            campaign_id, payload, settings=settings, client=client
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=409, detail=str(exc)) from exc
+    except YandexDirectError as exc:
+        diagnostics = exc.diagnostics or {}
+        detail: dict[str, Any] = {
+            "error_type": "YandexDirectError",
+            "message": str(exc),
+        }
+        if "error_code" in diagnostics:
+            detail["error_code"] = diagnostics["error_code"]
+        if "error_detail" in diagnostics:
+            detail["error_detail"] = diagnostics["error_detail"]
+        if "payload_preview" in diagnostics:
+            detail["payload_preview"] = diagnostics["payload_preview"]
+        raise HTTPException(status_code=502, detail=detail) from exc
+    except Exception as exc:
+        try:
+            store.append_audit(
+                "yandex_keyword_bids_failed",
+                campaign_id,
+                dry_run=False,
+                details={
+                    "campaign_id": campaign_id,
+                    "approved": payload.approved,
+                    "idempotency_key": payload.idempotency_key,
+                    "endpoint_safety_net": True,
+                    "yandex_error": (
+                        f"unexpected error in keyword bids endpoint: "
+                        f"{type(exc).__name__}: {exc}"
+                    ),
+                    "exception_type": type(exc).__name__,
+                },
+            )
+        except Exception:
+            pass
+        raise HTTPException(
+            status_code=502,
+            detail={
+                "error_type": "YandexDirectError",
+                "message": (
+                    f"unexpected error during keyword bids update: "
+                    f"{type(exc).__name__}"
+                ),
+            },
+        ) from exc
+
+
+# ---------------------------------------------------------------------------
+# Yandex Direct UTM — audit / plan / apply
+# ---------------------------------------------------------------------------
+
+
+@app.get(
+    "/yandex/campaigns/{campaign_id}/utm-audit",
+    response_model=UtmAuditResult,
+)
+def yandex_utm_audit(
+    campaign_id: str,
+    settings: Settings = Depends(get_settings),
+    client: YandexDirectClient | None = Depends(get_yandex_client),
+) -> UtmAuditResult:
+    """Read-only UTM audit for all ad and sitelink URLs in a campaign.
+
+    In mock mode, returns deterministic mock data. In sandbox/live modes,
+    reads real ads and sitelinks from Yandex Direct (no writes).
+    """
+    return store.utm_audit(
+        campaign_id,
+        settings=settings,
+        client=client,
+    )
+
+
+@app.post(
+    "/yandex/campaigns/{campaign_id}/utm-plan",
+    response_model=UtmPlanResult,
+)
+def yandex_utm_plan(
+    campaign_id: str,
+    payload: UtmPlanRequest,
+    settings: Settings = Depends(get_settings),
+    client: YandexDirectClient | None = Depends(get_yandex_client),
+) -> UtmPlanResult:
+    """Generate UTM plan/preview — always dry_run, never writes.
+
+    Returns the list of URL changes (old → new with UTM) and a
+    preview of the v5 ``ads.update`` payload that WOULD be sent on apply.
+    Sitelink previews are included but apply is not_implemented.
+    """
+    return store.utm_plan(
+        campaign_id,
+        payload,
+        settings=settings,
+        client=client,
+    )
+
+
+@app.post(
+    "/yandex/campaigns/{campaign_id}/utm-apply",
+    response_model=UtmApplyResult,
+)
+def yandex_utm_apply(
+    campaign_id: str,
+    payload: UtmApplyRequest,
+    settings: Settings = Depends(get_settings),
+    client: YandexDirectClient | None = Depends(get_yandex_client),
+) -> UtmApplyResult:
+    """Apply UTM URLs to live ads — write-gated.
+
+    * ``dry_run=True`` → preview only, ``applied=False``.
+    * ``dry_run=False`` requires:
+      1. ``DIRECTPILOT_MODE=live_write``
+      2. ``approved=true``
+      3. ``idempotency_key`` (>= 6 chars)
+
+    Uses ``ads.update`` (REPLACE-shaped) to safely update TextAd.Href.
+    Sitelink apply is not_implemented — surfaced in ``not_implemented``.
+    """
+    if not payload.approved:
+        raise HTTPException(
+            status_code=409,
+            detail="Action requires explicit approval before UTM apply",
+        )
+    if not payload.dry_run and settings.directpilot_mode != "live_write":
+        raise HTTPException(
+            status_code=409,
+            detail=(
+                f"Live writes require DIRECTPILOT_MODE=live_write; "
+                f"current mode is {settings.directpilot_mode!r}; "
+                f"UTM apply is not allowed in this mode "
+                f"(dry_run=True is the only allowed path)"
+            ),
+        )
+    try:
+        return store.utm_apply(
+            campaign_id,
+            payload,
+            settings=settings,
+            client=client,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=409, detail=str(exc)) from exc
+    except YandexDirectError as exc:
+        raise _yandex_error_to_502(exc) from exc
+
+
 # ---------------------------------------------------------------------------
 # Yandex Metrika — read-only (counters, goals, summary, traffic-sources)
 #