docs: add TypeScript external storage snipsync snippets - #855
Open
lennessyy wants to merge 6 commits into
Open
Conversation
Adds the TypeScript counterparts to the Python (#798) and Go (#804) external storage snippets, so the TypeScript docs page can pull CI-checked code instead of hand-maintained blocks. Snippets: typescript-s3-driver-create typescript-gcs-driver-create typescript-s3-external-storage-setup typescript-custom-storage-driver typescript-custom-driver-data-converter typescript-external-storage-threshold typescript-external-storage-multiple-drivers The GCS pair has no Python or Go equivalent; the TypeScript SDK is currently the only one shipping a first-party GCS driver. Two supporting changes: Bump the core @temporalio/* dependencies from ^1.18.1 to ^1.21.1. The external storage API landed in 1.21, and the driver packages pin @temporalio/common exactly, so the older floor produced a duplicate nested copy of common and type identity errors. Exclude features/snippets/external_storage from the per-SDK-version harness build in sdkbuild/typescript.go. That build generates its own package.json containing only the core SDK packages at the version under test, so it cannot resolve the external-storage-* drivers, and those drivers are versioned independently of the SDK anyway. The snippets are still fully type-checked by the root tsconfig.json in the `build-typescript` CI job, which is where the driver packages are declared.
lennessyy
added a commit
to temporalio/documentation
that referenced
this pull request
Jul 27, 2026
Wraps the seven substantial code blocks in SNIPSTART/SNIPEND markers pointing at the snippets added in temporalio/features#855, matching how the Python and Go pages source their samples. Short blocks stay inline, the same ones the Python and Go pages keep inline: the npm install commands, the SigV4A signer import, and the MRAP bucket example. The markers are inert until features#855 merges. Snipsync skips ids it cannot resolve, verified by a local run that left this page byte identical, so the two PRs can land in either order.
The TypeScript External Storage docs page now pulls its custom-driver example from the external-storage sample in samples-typescript, which runs and tests the driver end to end. Keeping a second copy here would give the same two snipsync ids two sources. The S3, GCS, threshold, and multiple-driver snippets stay, since the sample covers only the custom-driver path and has no S3 or GCS code. Python and Go custom-driver snippets are untouched.
…s' into docs/external-storage-snippets-ts
The docs pages render these regions verbatim, so a reader copying a snippet had no way to tell which package each symbol came from. Three of the packages involved have near-identical names, and ExternalStorage and Worker were not named anywhere else on the page. Move the eslint-disable comment above the marker so the region can start at the imports, since import/first forbids any statement preceding them. Drop the block-scope wrappers in the driver-create snippets, which the region no longer needs, and move the driver scaffold in the setup snippet below SNIPEND. Top-level await is unavailable here because these files compile as CJS, so the setup snippet keeps its async wrapper.
createClientAndWorker matches the surrounding prose, which tells the reader to pass the converter to their Client and Worker. The wrapper is the only scaffolding visible in the rendered snippet, so it should read like something a reader would write.
lennessyy
added a commit
to temporalio/documentation
that referenced
this pull request
Jul 30, 2026
Bump snipsync to 1.13.0, which indents spliced blocks to match their SNIPSTART marker instead of writing at column 0. A fenced block at column 0 inside a numbered list closes the list item, which split the Go and Python procedures into two lists and broke the MDX build outright on the TypeScript page, where the marker sits inside a Tabs element. That is the whole change to the Go and Python pages: same content, now indented under their markers. Two Python markers were at six spaces while their SNIPEND and list content were at three, so normalize them. The TypeScript snippets were materialized by pinning the snipsync origins at temporalio/features#855 and temporalio/samples-typescript#503, then restoring the config. A run against main is a no-op, so the daily snipsync job will re-sync these byte for byte once both land. Until they do, the source files are not on main, so the generated code block titles and the links to the samples repo would 404. Strip them, along with the custom driver excerpt that came from the unmerged sample. The daily job restores the titles on its own after the sources land.
lennessyy
added a commit
to temporalio/documentation
that referenced
this pull request
Jul 30, 2026
* Initial ts docs for extstore. * Remove </content></invoke> * fix(external-storage): correct TypeScript API and move into data-handling Rewrite the TypeScript External Storage page against the shipped API in @temporalio/*@1.21.1. Every code sample now typechecks against the published packages. API corrections: - Install @temporalio/external-storage-s3-aws-sdk alongside the driver; S3StorageDriver takes an S3StorageDriverClient, not an S3Client. - ExternalStorage is a class, not an object literal. - StorageDriverClaim is a class taking claimData positionally. - StorageDriver exposes readonly name and type properties rather than a name() method. Document type, which was missing. - context.target is a discriminated union keyed on kind, with the identifier in id. - The AWS SDK for JavaScript does not bundle a SigV4A signer, so MRAP requires installing and importing @aws-sdk/signature-v4a. The previous text described Go SDK behavior and would have left MRAP requests failing. Also adds a Google Cloud Storage section, maxPayloadSize, the required IAM permissions, and the duplicate driver name constraint. Structural: main restructured TypeScript data handling in #4928, so move the page under best-practices/data-handling, fix two links to the removed converters-and-encryption page, and drop the slug to match its siblings. Add the page to the sidebar, the data-handling index table, and the five Go/Python references in the External Storage encyclopedia page. * docs(external-storage): use ReleaseNoteHeader instead of info admonitions Replace the ":::info Release, stability, and dependency info" admonition with the ReleaseNoteHeader component on all four External Storage pages (Go, Python, TypeScript, and the encyclopedia). The component renders the "Currently in: Public Preview" pill and links to the release stages page, so the sentence naming the release stage is dropped from the body. The remaining copy carries the API stability caveat and the Slack channel link. Register externalStorage in FEATURE_RELEASE_TYPES so both the component and scripts/mdx-to-md.mjs resolve the label from one place. * docs(external-storage): blend S3 and GCS setup into one tabbed section The GCS section read as an abbreviated version of the S3 one. Merge both into a single "Store and retrieve large payloads" flow with backend tabs on the two things that actually differ, the install command and the driver construction. The ExternalStorage and Data Converter step is shared, so it is now written once instead of being summarized for GCS. Keep the full backend names in the H2 for search, and pin a short "#store-and-retrieve-large-payloads" anchor so the heading can grow to a third driver without breaking inbound links. Repoint the two encyclopedia deep links at that anchor. Also drop the prerequisites paragraph explaining how the S3 driver packages are split. Prerequisites should say what to install, not describe package internals. * docs(external-storage): wire TypeScript code samples to snipsync Wraps the seven substantial code blocks in SNIPSTART/SNIPEND markers pointing at the snippets added in temporalio/features#855, matching how the Python and Go pages source their samples. Short blocks stay inline, the same ones the Python and Go pages keep inline: the npm install commands, the SigV4A signer import, and the MRAP bucket example. The markers are inert until features#855 merges. Snipsync skips ids it cannot resolve, verified by a local run that left this page byte identical, so the two PRs can land in either order. * docs(external-storage): retarget custom-driver snippets at the samples repo The custom-driver example now comes from the external-storage sample in samples-typescript, which runs the driver end to end and tests it, instead of a hand-written copy in the features repo. The sample's FileSystemStorageDriver is a better example than the one it replaces: it content-addresses payloads so retries don't accumulate duplicate blobs, writes atomically, verifies the hash on read, rejects claims that resolve outside the storage root, and stores payloads concurrently while honoring the SDK's abort signal. Also corrects the driver `type` example. The built-in GCS driver reports "gcp.gcsdriver", which the page never mentioned. The S3 and GCS snippets still come from the features repo; the sample has no S3 or GCS code. * docs(external-storage): correct the payload size threshold boundary All three SDKs offload a payload when its serialized size is greater than or equal to the threshold, not strictly greater: - TypeScript: external-storage-runner.ts, `if (size < payloadSizeThreshold) continue` - Python: _extstore.py, `if payload.ByteSize() < self.payload_size_threshold` - Go: internal_extstore.go, `if proto.Size(p) < v.params.payloadSizeThreshold` The pages said "larger than 256 KiB", and the TypeScript page said payloads "at or below the threshold stay inline", which is backwards at the boundary. The Python SDK's own S3 driver README already documented this correctly. Also notes that the size compared is the serialized Payload including its metadata, not the raw application data. Separately, the encyclopedia told all readers to set the threshold to zero to externalize every payload. That is right for Python and TypeScript but wrong for Go, where zero selects the 256 KiB default and 1 is the value that offloads everything. The Go SDK page already had this right. * docs(external-storage): shorten the custom-driver excerpt The whole FileSystemStorageDriver class was too long to read inline. Show just store and retrieve, which is what the surrounding prose walks through, and link the sample for the per-payload work in the private helpers: content-addressed keys, the atomic write, the hash check on read, and the guard against claims that resolve outside the storage root. Markers narrowed in samples-typescript to match. * docs(external-storage): wire TypeScript snippets to snipsync Bump snipsync to 1.13.0, which indents spliced blocks to match their SNIPSTART marker instead of writing at column 0. A fenced block at column 0 inside a numbered list closes the list item, which split the Go and Python procedures into two lists and broke the MDX build outright on the TypeScript page, where the marker sits inside a Tabs element. That is the whole change to the Go and Python pages: same content, now indented under their markers. Two Python markers were at six spaces while their SNIPEND and list content were at three, so normalize them. The TypeScript snippets were materialized by pinning the snipsync origins at temporalio/features#855 and temporalio/samples-typescript#503, then restoring the config. A run against main is a no-op, so the daily snipsync job will re-sync these byte for byte once both land. Until they do, the source files are not on main, so the generated code block titles and the links to the samples repo would 404. Strip them, along with the custom driver excerpt that came from the unmerged sample. The daily job restores the titles on its own after the sources land. --------- Co-authored-by: Duncan Mackenzie <duncanma@duncanmackenzie.net> Co-authored-by: Lenny Chen <lenny.chen@temporal.io> Co-authored-by: Lenny Chen <55669665+lennessyy@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
Adds the TypeScript counterparts to the Python (#798) and Go (#804) external storage snippets, so the TypeScript docs page can pull CI-checked code instead of hand-maintained blocks.
typescript-s3-driver-creates3_setup/s3_driver_create.tstypescript-s3-external-storage-setups3_setup/s3_external_storage_setup.tstypescript-gcs-driver-creategcs_setup/gcs_driver_create.tstypescript-custom-storage-drivercustom_driver/custom_storage_driver.tstypescript-custom-driver-data-convertercustom_driver/custom_driver_data_converter.tstypescript-external-storage-thresholdthreshold/threshold_config.tstypescript-external-storage-multiple-driversmultiple_drivers/multiple_drivers.tsThe GCS pair has no Python or Go equivalent. The TypeScript SDK is currently the only one shipping a first-party GCS driver (
@temporalio/external-storage-gcs).Short blocks stay inline in the docs, matching how the Python and Go pages handle
npm install/ MRAP examples.Supporting changes worth a look
Bumped core
@temporalio/*from^1.18.1to^1.21.1. The external storage API landed in 1.21, and the driver packages pin@temporalio/commonexactly. With the older floor, npm installed a second nested copy ofcommonand the snippets failed with type identity errors.Excluded
features/snippets/external_storagefrom the per-SDK-version harness build (sdkbuild/typescript.go). That build generates its ownpackage.jsoncontaining only the core SDK packages at the version under test, so it cannot resolve@temporalio/external-storage-*. Adding them there is not workable either, since the driver packages pin an exactcommonversion and would conflict whenever the version under test differs.The snippets are still fully type-checked, just by the root
tsconfig.jsonin thebuild-typescriptjob, which is where the driver packages are declared. Verified thatgo run . prepare --lang tsstill compilessnippets/pluginsandsnippets/workerwhile skippingexternal_storage.Verification
npx tsc --buildpassesnpm run lintpasses (eslint + prettier)go build ./sdkbuild/ ./cmd/ .passes,gofmtcleango run . prepare --lang ts --version 1.21.1completes itstsc --buildThe snippet code is not speculative. Every block was run end to end against a real S3 bucket in
us-east-2first: offload above the 256 KiB threshold, inline below it,payloadSizeThreshold: 0, the custom disk driver, and multi-driver selector routing.