Merge branch 'main' into maplexu/AI-141-langsmith-docs

Author: brianmacdonald-temporal

docs/cloud/high-availability/ha-connectivity.mdx

@@ -3,16 +3,84 @@ id: ha-connectivity
 title: Connectivity for High Availability
 sidebar_label: Connectivity for High Availability
 slug: /cloud/high-availability/ha-connectivity
-description: How to use private network connectivity with Temporal Cloud HA features
+description: How to choose endpoints and configure private network connectivity for Namespaces with Temporal Cloud High Availability features.
 ---
 
 import { CaptionedImage, JsonTable } from '@site/src/components';
 
-:::tip Namespaces with High Availability features and private connectivity
+A Namespace with High Availability features spans two regions, and the endpoint your Workers and Clients connect through determines how they behave before, during, and after a failover.
+This page covers:
 
-Proper networking configuration is required for failover to be transparent to clients and Workers when using AWS PrivateLink or GCP Private Service Connect.
+- How to choose between the Namespace Endpoint and a Regional Endpoint for a Namespace with High Availability features.
+- How to configure PrivateLink so that failover remains transparent to Workers on private networks.
 
-This page covers single-cloud HA (both replicas on AWS, or both on GCP) and multi-cloud HA (one replica on AWS, one on GCP).
+## How to choose an endpoint for a Namespace with High Availability features
+
+Temporal Cloud exposes two kinds of gRPC endpoints for a Namespace.
+See [How to access a Namespace](/cloud/namespaces#access-namespaces) for the general definitions; this section focuses on how each behaves with replication and failover.
+
+### Namespace Endpoint (recommended)
+
+Format: `<namespace>.<account>.tmprl.cloud:7233`
+
+The Namespace Endpoint always connects to whichever region is currently active.
+Under the hood, it is a CNAME that points at the active region's Regional Endpoint.
+When Temporal Cloud fails the Namespace over, it updates the CNAME to point at the new active region.
+The DNS <a href="https://en.wikipedia.org/wiki/Time_to_live">TTL</a> is 15 seconds, so Clients converge within about 30 seconds with no configuration change on your side.
+
+Use the Namespace Endpoint unless you have a specific reason to pin traffic to a region.
+
+### Regional Endpoint
+
+Format: `<cloud>-<region>.region.tmprl.cloud:7233` (for example, `aws-us-west-2.region.tmprl.cloud` or `gcp-us-central1.region.tmprl.cloud`).
+See [regions](/cloud/regions) for the full list.
+
+A Regional Endpoint is shared across every Namespace that is active or replicated in that region.
+Unlike the Namespace Endpoint, a Regional Endpoint stays pinned to the region in its name — if that region holds the passive replica of your Namespace, the Regional Endpoint connects to the passive replica.
+
+Use a Regional Endpoint only when you need explicit control over which replica a Client or Worker reaches.
+
+Trade-offs to consider:
+
+- **Faster recovery.** A Worker connecting through a Regional Endpoint skips the DNS step that Clients on the Namespace Endpoint wait for during a failover. This removes the ~30-second DNS convergence window from the recovery path, which is useful for Workloads that must minimize [Recovery Time](/cloud/rpo-rto) at all costs.
+- **You are responsible for regional coverage.** A Worker using the Regional Endpoint of a region cannot reach the Namespace if that region is in an outage. To stay available through a failover, you must run Workers that use the **replica** region's Regional Endpoint — Workers pointed only at the outage region's Regional Endpoint will not reconnect automatically.
+
+When authenticating with mTLS, set the Client's `server_name` / `serverNameOverride` config equal to the Namespace Endpoint.
+This overrides the SNI that the Client will expect during the TLS Handshake with Temporal Cloud.
+The Regional Endpoint forwards the request to your Namespace, so the Client must expect the Namespace's certificate during the TLS handshake.
+
+For example, in Typescript, the Client's config would be set like this:
+
+```typescript
+await Connection.connect({
+  address: 'aws-us-east-1.region.tmprl.cloud:7233',
+  tls: {
+    serverNameOverride: 'my-namespace.my-account.tmprl.cloud',
+    clientCertPair: { crt: clientCert, key: clientKey },
+  },
+  ...
+});
+```
+
+### How endpoints route on failover
+
+Consider a Namespace replicated across `us-east-1` (initially active) and `us-west-2` (initially the replica), with a failover that swaps the two.
+
+| Client connects via                         | Before failover | After failover          |
+| ------------------------------------------- | --------------- | ----------------------- |
+| Namespace Endpoint                          | `us-east-1`     | `us-west-2` (automatic) |
+| Regional Endpoint `aws-us-east-1.region...` | `us-east-1`     | `us-east-1`             |
+| Regional Endpoint `aws-us-west-2.region...` | `us-west-2`     | `us-west-2`             |
+
+The Namespace Endpoint moves with the active region via an updated CNAME — no Client changes required.
+The Regional Endpoints do not change their targets on failover: each continues to route to the replica that lives in its region.
+
+## How to use PrivateLink with High Availability features
+
+:::tip
+
+Proper networking configuration is required for failover to be transparent to Clients and Workers when using PrivateLink.
+This section describes how to configure routing for Namespaces with High Availability features on AWS PrivateLink.
 
 :::
 
@@ -32,37 +100,37 @@ Namespace DNS records have a 15-second TTL. Clients should converge to the new r
 
 For private connectivity, your job is to make sure that:
 
-1. Both regions resolve to the correct private endpoint inside your network — not the public internet.
-2. Your Workers have a network path to whichever region becomes active.
+- Override the Regional Endpoint's DNS zone to resolve to a VPC Endpoint.
+- Ensure network connectivity between the two regions.
 
 ## Single-cloud HA on AWS PrivateLink
 
-This is the most common setup: both replicas live in AWS regions, and Workers connect via AWS PrivateLink.
+### How Namespace DNS records work with PrivateLink
 
 When using PrivateLink, you connect to Temporal Cloud through a VPC Endpoint, which uses addresses local to your network.
-Temporal treats each `region.tmprl.cloud` zone as a separate zone, so you override resolution per region.
+Temporal treats each `region.<tmprl_domain>` as a separate zone.
+This setup allows you to override the default zone, ensuring that traffic is routed internally for the regions you're using.
 
-Before failover, with the active region being `aws-us-west-2`:
+A Namespace's active region is reflected in the target of the Namespace Endpoint's CNAME record.
+For example, if the active region of a Namespace is AWS us-east-1, the DNS configuration would look like this:
 
-| Record name                         | Record type | Value                            |
-| ----------------------------------- | ----------- | -------------------------------- |
-| ha-namespace.account-id.tmprl.cloud | CNAME       | aws-us-west-2.region.tmprl.cloud |
+| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-east-1.region.tmprl.cloud |
+| ----------------------------------- | ----- | -------------------------------- |
 
-After a failover to `aws-us-east-1`, Temporal Cloud rewrites the CNAME:
+After a failover, the CNAME record is updated to point to the failover region, for example:
 
-| Record name                         | Record type | Value                            |
-| ----------------------------------- | ----------- | -------------------------------- |
-| ha-namespace.account-id.tmprl.cloud | CNAME       | aws-us-east-1.region.tmprl.cloud |
+| ha-namespace.account-id.tmprl.cloud | CNAME | aws-us-west-2.region.tmprl.cloud |
+| ----------------------------------- | ----- | -------------------------------- |
 
-The Temporal-managed CNAME changed from us-west-2 to us-east-1 — your private DNS does not need to change.
+The Temporal domain did not change, but the CNAME updated from us-east-1 to us-west-2.
 
 <CaptionedImage
     src="/img/cloud/high-availability/private-link.png"
     title="Customer side solution example"
     zoom="true"
 />
 
-### Setting up the DNS override (AWS)
+### How to set up the DNS override
 
 In AWS, use a Route 53 private hosted zone for `region.tmprl.cloud` to override resolution per region:
 
@@ -114,24 +182,18 @@ If you run Workers on Alpine and use multi-cloud HA, either:
 
 :::
 
-## Test failover before you depend on it
-
-Failover is the only thing High Availability features exist to do — and DNS, cross-region or cross-cloud reachability, and Connectivity Rule coverage are exactly the kinds of configuration that look correct on paper and break under failover. Test it in a non-production Namespace first.
+To set up the DNS override, configure specific regions to target the internal VPC Endpoint IP addresses.
+For example, you might set `aws-us-west-1.region.tmprl.cloud` to target `192.168.1.2`.
+In AWS, this can be done using a Route 53 private hosted zone for `region.tmprl.cloud`.
+Link that private zone to the VPCs you use for Workers.
 
 A reasonable validation plan:
 
-1. Set up the HA Namespace and the private connectivity for both regions, including all DNS overrides.
-2. Run Workers continuously in **both** regions (or arrange cross-region connectivity).
-3. Trigger a manual failover from the Web UI or `tcld` and verify:
-   - DNS for `<ns>.<account>.tmprl.cloud` resolves to the new region within ~30 seconds.
-   - Workers in both regions are picking up tasks.
-   - SDK clients connect successfully (no `Name resolution failed`, `connection reset by peer`, or `context deadline exceeded` errors).
-4. Trigger a failback to the original region and verify the same.
-5. For multi-cloud HA, repeat with each cloud as the active replica, including from base images (Alpine, distroless) you actually use in production.
-
-If a real failover finds a configuration gap that wasn't tested, recovery typically requires changes on the client side that are hard to make under pressure.
+Consider how you'll configure Workers for this setup.
+You can either have Workers run in both regions continuously or establish connectivity between regions using Transit Gateway or VPC Peering.
+Either approach ensures Workers can access the newly activated region once failover occurs.
 
-## Available regions, PrivateLink endpoints, and DNS record overrides
+### Available regions, PrivateLink endpoints, and DNS record overrides
 
 :::caution
 

docs/cloud/migrate/automated.mdx

@@ -289,6 +289,12 @@ Temporal will generate the endpoint-id and initiate the migration. During this p
 while the cloud Namespace becomes passive. Workflows are replicated from the self-hosted Namespace to the cloud Namespace. Once the
 cloud Namespace has fully synced with self-hosted Namespace, migration is ready for handover.
 
+:::note Billing
+
+Billing for the cloud Namespace does not begin until the migration is [confirmed](#confirm-complete).
+
+:::
+
 The following [command](https://pkg.go.dev/github.com/temporalio/tcld#readme-start-a-migration) is used to start the
 migration:
 
@@ -341,7 +347,7 @@ Use the following checklist prior to finalizing the migration:
   lead to unexpected symptoms and optimizations.
 - Know how to reach out to your Temporal Solutions Architect (SA) and Account Executive (AE) for assistance.
 
-### Confirm complete
+### Confirm complete {#confirm-complete}
 
 Once a Namespace has been transferred to the cloud and validated, the migration will be completed. Note that this step
 is final and may not be undone. Once performed, Workflow replication from the cloud Namespace to the self-hosted server

docs/develop/go/best-practices/data-handling/external-storage.mdx

@@ -13,8 +13,8 @@ description: Offload large payloads to external storage using the claim check pa
 
 :::info Release, stability, and dependency info
 
-External Storage is in [Pre-Release](/evaluate/development-production-features/release-stages#pre-release). APIs and
-configuration may change before the stable release. Join the
+External Storage is in [Public Preview](/evaluate/development-production-features/release-stages#public-preview). APIs and
+configuration may change before General Availability. Join the
 [#large-payloads Slack channel](https://temporalio.slack.com/archives/C09VA2DE15Y) to provide feedback or ask for help.
 
 :::
@@ -34,6 +34,7 @@ The Go SDK includes an S3 storage driver. Follow these steps to set it up:
 
 - An Amazon S3 bucket that you have read and write access to. Refer to [lifecycle management](/external-storage#lifecycle)
   to ensure that your payloads remain available for the entire lifetime of the Workflow.
+  - The S3 driver has been validated with single-region buckets. Multi-region configurations such as AWS Multi-Region Access Points (MRAP) have not been validated.
 - Install the S3 driver module and its dependencies: `go get go.temporal.io/sdk/contrib/aws/s3driver go.temporal.io/sdk/contrib/aws/s3driver/awssdkv2 github.com/aws/aws-sdk-go-v2/config github.com/aws/aws-sdk-go-v2/service/s3`
 
 ### Procedure
@@ -87,6 +88,8 @@ w := worker.New(c, "my-task-queue", worker.Options{})
    All Workflows and Activities running on the Worker use the storage driver automatically without changes to your
    business logic. The driver uploads and downloads payloads concurrently and validates payload integrity on retrieve.
 
+   The S3 driver includes diagnostic metadata, such as the AWS region, in error messages to help troubleshoot storage failures.
+
 ## Implement a custom storage driver
 
 If you need a storage backend other than what the built-in drivers allow, you can implement your own storage driver.