|
| 1 | +--- |
| 2 | +title: Cost governance on Temporal Cloud |
| 3 | +sidebar_label: Cost Governance |
| 4 | +description: A design framework for attributing Temporal Cloud spend to teams, forecasting budget burn rate, and detecting usage anomalies using the Billing API, Billing Center, Usage Dashboards, and OpenMetrics. |
| 5 | +toc_max_heading_level: 4 |
| 6 | +tags: |
| 7 | + - Best Practices |
| 8 | + - Temporal Cloud |
| 9 | +--- |
| 10 | + |
| 11 | +This guide lays out a cost-governance framework built entirely on existing Temporal Cloud tooling: the |
| 12 | +[Cloud Billing API](/cloud/billing-api), [Billing Center](/cloud/billing), [Usage Dashboards](/cloud/actions-usage), |
| 13 | +and the [Action metric](/cloud/metrics/openmetrics/metrics-reference#temporal_cloud_v1_billable_action_count). |
| 14 | +Enterprise platform teams consistently need to solve three problems: |
| 15 | + |
| 16 | +- **Cost attribution**: attributing spend to the team, environment, and workload that generated it. |
| 17 | +- **Budget forecasting and trend tracking**: knowing where a Namespace stands against its monthly budget before the |
| 18 | + invoice closes, not after. |
| 19 | +- **Anomaly detection on usage**: treating a spend spike as an early operational signal, not just a finance surprise. |
| 20 | + |
| 21 | +This guide helps you build cost visibility in from the start, rather than discover the gap after costs have already |
| 22 | +accumulated. |
| 23 | + |
| 24 | +## Key tools and concepts |
| 25 | + |
| 26 | +Temporal provides tooling across Temporal Cloud, metrics, and the Cloud Operations API to address each of these use |
| 27 | +cases programmatically: |
| 28 | + |
| 29 | +| Tool | Grain | Best for | Who sees it | |
| 30 | +| :--- | :--- | :--- | :--- | |
| 31 | +| [Billing Center](/cloud/billing) | Monthly invoice | Summary invoices, credits, plan management | Account Owner, Finance Admin | |
| 32 | +| [Billing API](/cloud/billing-api) | Hourly, daily, monthly | Namespace and tag cost attribution, FinOps ingestion | Account Owner, Finance Admin | |
| 33 | +| [Usage Dashboards](/cloud/actions-usage) | Namespace, by Action category | At-a-glance usage in the Cloud UI | Account Owners, Finance Admins, and Global Admins at account level; anyone with Namespace access at Namespace level | |
| 34 | +| [Actions in Event History](/cloud/actions-usage#actions-in-workflows) | Per Workflow Execution | Estimating Actions for a specific execution | Account Owners, Global Admins, Namespace Admins, Developers, and Read-Only users | |
| 35 | +| [Action metric](/cloud/metrics/openmetrics/metrics-reference#temporal_cloud_v1_billable_action_count) | 1-minute, by Workflow and Action type | Near-real-time usage behavior, alerting, debugging | Service account with the Metrics Read-Only role | |
| 36 | + |
| 37 | +### Actions |
| 38 | + |
| 39 | +Temporal Cloud bills primarily on [Actions](/cloud/actions), which are billable operations such as starting a |
| 40 | +Workflow, scheduling an Activity, recording a Heartbeat, sending a Signal, or receiving a Query or Update. Actions are |
| 41 | +grouped into categories (Workflow, Activity, Timer, Signal, Query, Update, Schedule, Nexus) plus a few billed features |
| 42 | +(Export, Fairness, Capacity). For background on how usage-based pricing works on Temporal Cloud, see |
| 43 | +[Improved cost transparency with usage-based billing](https://temporal.io/blog/improved-cost-transparency-with-usage-based-billing). |
| 44 | + |
| 45 | +:::note |
| 46 | +Not every Action type appears in every surface. Billing and usage data has the most complete picture; Event History |
| 47 | +and OpenMetrics are best used for estimation and trend analysis, not invoice reconciliation. |
| 48 | +::: |
| 49 | + |
| 50 | +### Namespace tags |
| 51 | + |
| 52 | +The Billing API attributes every charge to a Namespace and enriches it with [tags](/cloud/namespaces#tag-a-namespace), |
| 53 | +user-defined key/value pairs such as team, environment, or cost center. The Billing API reads tags in real time, so a |
| 54 | +Namespace that shipped last month without a tag can be tagged today and have its historical cost re-attributed |
| 55 | +retroactively. This solves the most common early mistake: forgetting to tag a Namespace before it starts accruing |
| 56 | +spend. |
| 57 | + |
| 58 | +```json |
| 59 | +{ |
| 60 | + "$tmprl_project": ["claims-platform"], |
| 61 | + "team": ["map"], |
| 62 | + "env": ["prod"] |
| 63 | +} |
| 64 | +``` |
| 65 | + |
| 66 | +## Attribute cost by team and workload |
| 67 | + |
| 68 | +**Goal**: give every Temporal Cloud team, use case, and application (for example, Claim, Payment, Notification) its |
| 69 | +own line item, sourced directly from Temporal Cloud's billing data instead of an estimate. Use this for both cost |
| 70 | +attribution and budget tracking. |
| 71 | + |
| 72 | +### Design artifacts |
| 73 | + |
| 74 | +- Namespace-per-tenant-per-environment as the attribution boundary. Namespace is the finest-grained unit the Billing |
| 75 | + API attributes cost to. |
| 76 | +- A required tagging convention enforced at Namespace creation time. At minimum: team, env, and workload or product. |
| 77 | +- A daily pull of the Billing API report, joined on tags, feeding the internal FinOps or budget system so each tenant |
| 78 | + sees its own line item. |
| 79 | + |
| 80 | +**Recommended tagging taxonomy** |
| 81 | + |
| 82 | +| Tag key | Example value | Purpose | |
| 83 | +| :--- | :--- | :--- | |
| 84 | +| team | claim-team, payment-team, notification-team | Chargeback owner | |
| 85 | +| env | prd, stg, dev | Separates production spend from lower environments | |
| 86 | +| workload | claims-processing, payment, notification | Sub-team or product-line attribution | |
| 87 | +| cost-center | cc-12345 | Direct feed into the finance general ledger, if required | |
| 88 | + |
| 89 | +### Implementation path |
| 90 | + |
| 91 | +- Generate a [Billing API](/cloud/billing-api) report at daily granularity for the current and prior two billing |
| 92 | + months: create it with `CreateBillingReport`, poll `GetBillingReport`, then download the CSV. You can do this |
| 93 | + through the Cloud UI or the [Cloud Operations API](/ops). |
| 94 | +- Parse the FOCUS-aligned CSV. Group by `ResourceName` (Namespace name plus Temporal Cloud account ID) and `Tags`; |
| 95 | + sum `ContractedCost` by `ChargeDescription`. |
| 96 | +- Feed the grouped output into your internal FinOps or budget system as a recurring Workflow, or connect Temporal |
| 97 | + Cloud's native Datadog Cloud Cost Management or Vantage integration if either tool is already in your FinOps stack. |
| 98 | + |
| 99 | +## Forecast budget burn rate |
| 100 | + |
| 101 | +**Goal**: answer "where are we trending against this Namespace's budget, and will we exceed it before month end?" |
| 102 | +with enough lead time to act. |
| 103 | + |
| 104 | +### Two data sources, two jobs |
| 105 | + |
| 106 | +Budget tracking needs both a financially accurate source and a fast, directional source, because the Billing API's |
| 107 | +current-month data is provisional until the billing month closes: |
| 108 | + |
| 109 | +| Source | Accuracy | Latency | Role in burn-rate tracking | |
| 110 | +| :--- | :--- | :--- | :--- | |
| 111 | +| [Billing API](/cloud/billing-api) (daily grain) | Invoice-aligned | Usage up to current time minus 24 hours; final at month close | Month-to-date actual spend per Namespace, financial source of truth | |
| 112 | +| [Action metric](/cloud/metrics/openmetrics/metrics-reference#temporal_cloud_v1_billable_action_count) | Directional usage estimate | Available within about 3 minutes | Early-warning proxy for spend trajectory between billing pulls | |
| 113 | + |
| 114 | +### Burn-rate methodology |
| 115 | + |
| 116 | +- Maintain a budget table per Namespace (or per team, rolling up Namespaces) with a monthly allocation. |
| 117 | +- Each day, pull the Billing API daily report and compute month-to-date (MTD) actual spend per Namespace. |
| 118 | +- Project end-of-month (EOM) spend using a simple run-rate extrapolation: EOM projection = MTD spend ÷ days |
| 119 | + represented in the report × days in month. Derive the represented period from the report's charge dates so the |
| 120 | + roughly 24-hour Billing API lag doesn't depress the projection. |
| 121 | +- Compare the projection against the allocated budget and flag any Namespace trending to exceed it. |
| 122 | +- Between daily Billing API pulls, use the 24-hour Actions estimate from [OpenMetrics](/cloud/metrics/openmetrics) as |
| 123 | + a same-day directional check on whether a Namespace's usage trajectory has shifted: |
| 124 | + |
| 125 | +``` |
| 126 | +# 24-hour Actions estimate from OpenMetrics (PromQL) |
| 127 | +sum( |
| 128 | + avg_over_time( |
| 129 | + temporal_cloud_v1_billable_action_count{temporal_namespace="$namespace"}[24h:1m] |
| 130 | + ) |
| 131 | +) * 86400 |
| 132 | +``` |
| 133 | + |
| 134 | +### Alerting |
| 135 | + |
| 136 | +- Trigger a budget alert when a Namespace's EOM projection exceeds 90% (warning), 100% (target), or 110% |
| 137 | + (over budget) of its allocation. |
| 138 | +- Route alerts to the Namespace owner and the FinOps or platform DRI. |
| 139 | + |
| 140 | +:::important |
| 141 | +`temporal_cloud_v1_billable_action_count` is a usage estimate, not your bill. It doesn't account for storage, support |
| 142 | +fees, pricing-tier calculations, some features (TRUs, Fairness), or rounding. Use it to catch trend shifts early in |
| 143 | +the month, and reconcile final numbers against the Billing API or Billing Center once the billing month closes. |
| 144 | +::: |
| 145 | + |
| 146 | +## Detect usage anomalies early |
| 147 | + |
| 148 | +**Goal**: treat a sudden spend or Action spike as an early operational signal for a runaway Workflow or bad deploy. |
| 149 | + |
| 150 | +### Why OpenMetrics is the right layer |
| 151 | + |
| 152 | +The `temporal_cloud_v1_billable_action_count` metric is broken down by both `action_type` and `temporal_workflow_type` |
| 153 | +at one-minute granularity. That combination answers the two questions that matter most during an incident: which |
| 154 | +Workflow is driving the spike, and what kind of Action it's doing more of. |
| 155 | + |
| 156 | +For example queries, see |
| 157 | +[Getting the most out of the Billable Action Count metric](https://temporal.io/blog/getting-the-most-out-of-the-billable-action-count-metric). |
| 158 | +For a real-world example of using this metric to validate metering, see |
| 159 | +[Dogfooding the Billable Actions metric](https://temporal.io/blog/dogfooding-the-billable-actions-metric-how-granular-observability-improved-our-metering-validation). |
| 160 | + |
| 161 | +### Defining and finding anomalies |
| 162 | + |
| 163 | +Establish a baseline of Actions consumption. Your workload may have seasonality across hours, days, and weeks, which |
| 164 | +makes setting alerts harder. Some starting points: |
| 165 | + |
| 166 | +- Alert if the number of Actions for an hour exceeds the expected maximum for all hours in a day. |
| 167 | +- Alert if the number of Actions for a day exceeds the expected maximum for all days in a month. |
| 168 | + |
| 169 | +When an alert triggers, or during root cause analysis, check monitoring dashboards for more nuanced anomalies, |
| 170 | +including: |
| 171 | + |
| 172 | +- New business generating more Action usage. Determine whether the pattern is a spike or steady growth that's now |
| 173 | + exceeding typical thresholds — if it's growth, it's time to establish a new baseline. |
| 174 | +- Correlation with deploys. Feed deploy markers from your CI/CD pipeline into the same Datadog or Grafana dashboard |
| 175 | + as your Action-rate charts; a spike that lines up with a deploy timestamp is a regression. |
| 176 | + |
| 177 | +## Guardrails to design around |
| 178 | + |
| 179 | +- `temporal_cloud_v1_billable_action_count` is a directional estimate. Don't use it for financial reconciliation — |
| 180 | + use the Billing API instead. |
| 181 | +- Current-month billing data is provisional. It finalizes only when the billing month closes, and available data |
| 182 | + lags current time by roughly 24 hours. |
| 183 | +- Only one billing report generates at a time per account; additional requests queue rather than fail. Use an |
| 184 | + idempotency key (`async_operation_id`) on retries and poll with exponential backoff. |
| 185 | +- High-cardinality metrics need filtering at scale. Plan Namespace and label filtering into your |
| 186 | + [OpenMetrics](/cloud/metrics/openmetrics) scrape configuration from day one, not after you hit the datapoint |
| 187 | + ceiling. |
| 188 | + |
| 189 | +None of the three use cases in this guide need new Temporal Cloud features. They need a deliberate design applied to |
| 190 | +tooling that already exists: a tagging convention enforced at Namespace creation, a scheduled pull of the Billing |
| 191 | +API, and an OpenMetrics-based alerting layer tuned to your account's scale. The organizations that get the most value |
| 192 | +from this treat it as a platform requirement before the first tenant onboards, not as a remediation project after the |
| 193 | +first unexplained invoice. |
| 194 | + |
| 195 | +The result is a Temporal Cloud bill that behaves like any other well-governed enterprise OpEx line: attributable, |
| 196 | +forecastable, and defensible. |
0 commit comments