Skip to content

functions.mdx

Latest commit

 

History

History
602 lines (439 loc) · 24.4 KB

functions.mdx

File metadata and controls

602 lines (439 loc) · 24.4 KB
Raw
title Resolver functions
description A reference of all available function resolvers in varlock

You may use resolver functions instead of static values within both config items and decorator values.

Functions can be composed together to create more complex value resolution logic.

ITEM=fn(arg1, arg2)
COMPOSITION=fn1(fn1Arg1, fn2(fn2Arg1, fn2Arg2))

Note that many built-in utility functions have expansion equivalents and often it will be more clear to use them that way. For example:

EXPANSION_EQUIVALENT="pre-${OTHER}-post"
USING_FN_CALLS=concat("pre-", ref(OTHER), "-post")

# mixed example
CONFIG=exec(`./scripts/load-config.sh ${APP_ENV}`)

There are built-in utility functions, random value generators, generateOtp() for 2FA codes, oauth() for exchanging long-lived OAuth credentials for fresh access tokens, a cache() function for reusing values according to your global cache mode, encryption functions for device-local secrets, and plugin-provided resolver functions that can fetch data from external providers. See the Plugins guide for more information on plugin-provided functions.

Core

### `ref()`

References another config item (env var) - which is useful when composing multiple functions together.

Expansion equivalent: ref(OTHER_VAR) === ${OTHER_VAR} (and also $OTHER_VAR)

We recommend using the bracketed version within string templates, and the simpler version when referencing an item directly.

API_URL=https://api.example.com
USERS_API_URL=${API_URL}/users
USERS_API_URL2=concat(ref("API_URL"), "/users") # without using expansion
### `concat()`

Concatenates multiple values into a single string.

Expansion uses concat() to combine multiple parts of strings when they include multiple parts.

PATH=concat("base/", ref("APP_ENV"), "/config.json")
PATH2=`base/${APP_ENV}/config.json` # equivalent using expansion
### `exec()`

Executes a CLI command and uses its output as the value. This is particularly useful for integrating with external tools and services.

:::note Many CLI tools output an additional newline. exec() will trim this automatically. :::

:::tip[Prefer a plugin when one exists] If varlock has a plugin for your provider (1Password, AWS, Vault, …), use its resolver (e.g. op()) instead of shelling out. Plugins handle auth, caching, and validation. Reach for exec() only for tools without a plugin, or for custom logic. :::

Expansion equivalent: exec(command) === $(command)

# A custom or internal secrets CLI
API_KEY=exec(`my-secrets-cli get api-key`)
# Any command whose stdout becomes the value
GIT_SHA=exec(`git rev-parse HEAD`)
### `fallback()`

Returns the first non-empty value in a list of possible values.

POSSIBLY_EMPTY=
ANOTHER=
EXAMPLE=fallback(ref(POSSIBLY_EMPTY), ref(ANOTHER), "default-val")
### `remap()`

Maps a value to a new value based on a set of lookup pairs. This is useful for translating one value, often provided by an external platform, into another.

  • The first argument is the value to remap (often a ref() to another variable).
  • All following arguments are pairs of (matchValue, resultValue).
  • An optional trailing default value can be added as the last argument (when the total number of remaining args is odd).
  • Match values can be a string, undefined, or a regex-like string (/pattern/).
  • If no match is found and there is no default, the original value is returned.
# env var that is set by CI/platform
CI_BRANCH=

# @type=enum(development, preview, production)
APP_ENV=remap($CI_BRANCH, "main", production, /.*/, preview, undefined, development)

:::caution[Quoting path-like values] An unquoted match value that looks like a valid regex (i.e., /something/ with optional flags) will be treated as a regex pattern. If you need to match a literal string containing slashes (like a file path), wrap it in quotes.

# /usr/local/ looks like a valid regex, so use quotes to match it literally
ITEM=remap($PATH_VAR, "/usr/local/", found, default)

:::

:::note[Deprecated syntax] The old key=value syntax (result=match) is still supported but deprecated. Use positional pairs instead.

# deprecated - key is the result, value is what to match (backwards and limited by key naming)
APP_ENV=remap($CI_BRANCH, production="main", preview=/.*/, development=undefined)

:::

Utilities

### `ifs()`

Evaluates a series of condition/value pairs, returning the value for the first truthy condition. Similar to Excel's IFS function.

  • Arguments are pairs of (condition, value).
  • An optional trailing default value can be added as the last argument (when the total number of args is odd).
  • If no condition is truthy and there is no default, returns undefined.
ENV=staging

# returns the value matching the first truthy condition
API_URL=ifs(
  eq($ENV, production), https://api.example.com,
  eq($ENV, staging), https://staging-api.example.com,
  http://localhost:3000
)
### Regex-like strings

Certain functions like remap() and type options like matches support regex pattern matching. You can use JavaScript-style regex syntax (/pattern/flags) as an unquoted value. These will be automatically detected and treated as regular expressions.

A string is treated as a regex when it:

  • Starts and ends with / (with optional flags like i, g, m, s, u, y after the closing /)
  • Is not wrapped in quotes
# regex pattern in remap, matches case-insensitively
ENV_TYPE=remap($APP_ENV, /^dev.*/i, dev, "production", prod)

# regex pattern in type options
# @type=string(matches=/^sk-[a-zA-Z0-9]+$/)
API_KEY=

# @type=url(matches=/^https:\/\/api\./)
API_URL=

:::caution[Paths vs regex ambiguity] Since /something/ looks like a valid regex, values like /usr/lib/ will be treated as regex patterns in contexts that support them (like remap() match values). To use a literal string containing slashes, wrap it in quotes:

# unquoted, treated as regex pattern matching "usr/lib"
ITEM=remap($VAR, /usr/lib/, matched)

# quoted, treated as the literal string "/usr/lib/"
ITEM=remap($VAR, "/usr/lib/", matched)

Note that as a top-level config value (e.g., MY_PATH=/usr/local/bin), slash-containing strings are always treated as plain strings. :::

:::note[regex() function] The older regex("pattern") function wrapper is still supported but the /pattern/ literal syntax is preferred, since it's more concise and supports flags.

# older style - still works
ENV_TYPE=remap($APP_ENV, regex("^dev.*"), dev, "production", prod)

:::

### `forEnv()`

Resolves to a boolean, if the current environment matches any in the list passed in as args.

Requirements:

  • Requires an @currentEnv to be set in your .env.schema file
  • Takes one or more environment names as arguments
# @currentEnv=$APP_ENV @defaultRequired=false
# @disable=forEnv(test)  # entire file will be disabled if env is test
# ---
APP_ENV=staging

# Required only in development
# @required=forEnv(development)
DEV_API_KEY=

# Required in staging and production
# @required=forEnv(staging, production)
PROD_API_KEY=
### `eq()`

Checks if 2 values are equal and resolves to a boolean.

IS_STAGING_DEPLOYMENT=eq($GIT_BRANCH, "staging")
### `if()`

Checks a boolean to return a true/false option

API_URL=if(eq($GIT_BRANCH, "main"), api.example.com, staging-api.example.com)
### `not()`

Negates a value and returns a boolean. Falsy values are - false, "", 0, undefined, and will be negated to true. Otherwise will return false.

# Negate the result of another function
SHOULD_DISABLE_FEATURE=not(forEnv(production))
### `isEmpty()`

Returns true if the value is undefined or an empty string, false otherwise.

# Check if a value is empty
HAS_API_KEY=not(isEmpty($API_KEY))

# Use with conditional logic
API_URL=if(isEmpty($CUSTOM_API_URL), "https://api.default.com", $CUSTOM_API_URL)

Random value generators

These functions generate random values using cryptographically secure randomness (node:crypto). In ephemeral environments, it can be helpful to generate values that are unique per run/deployment. For local dev, the cache() function can be used to keep a value stable for a period of time.

### `randomNum()`

Generates a random number. Integer by default; if you pass precision=N, returns a float with N decimal places.

  • With 1 arg: generates between 0 and max (inclusive)
  • With 2 args: generates between min and max (inclusive)
  • precision=N option switches to float mode (decimal places, 0-20)
# Cached random port between 3000 and 4000 (integer)
DEV_PORT=cache(randomNum(3000, 4000))

# One-off random integer up to 1000
SEED=randomNum(1000)

# Random float between 0 and 1 with 4 decimal places
RATE=randomNum(0, 1, precision=4)

# Cached random float between 10 and 20 with 4 decimal places
THRESHOLD=cache(randomNum(10, 20, precision=4))
### `randomUuid()`

Generates a random UUID v4.

# Unique identifier for this environment (stable across runs)
INSTANCE_ID=cache(randomUuid())

# Per-run / per-evaluation ID
REQUEST_ID=randomUuid()
### `randomHex()`

Generates a random hexadecimal string. By default the argument is the character length of the output. Pass bytes=true to interpret the argument as a byte count instead (where each byte = 2 hex characters). Default is 32 characters.

# 32-character hex string (default)
# @sensitive
SESSION_SECRET=cache(randomHex())

# 64-character hex string
# @sensitive
ENCRYPTION_KEY=cache(randomHex(64))

# 32 bytes (= 64 hex chars), byte-length mode
# @sensitive
HMAC_KEY=cache(randomHex(32, bytes=true))

# Non-sensitive one-off value
NONCE=randomHex(16)
### `randomString()`

Generates a random alphanumeric string. Default length is 16 characters using A-Za-z0-9.

  • First arg: character length (default: 16)
  • charset=S option: custom character set to draw from
# 32-character alphanumeric string
# @sensitive
API_SECRET=cache(randomString(32))

# 8-character string from custom charset
PIN_CODE=cache(randomString(8, charset="0123456789"))

# One-off random string
TEMP_LABEL=randomString(10)

One-time passwords

### `generateOtp()`

Generates a time-based one-time password (TOTP) code from a shared secret, the same codes an authenticator app shows. Useful for CLIs that require a 2FA code on every invocation, e.g. aws sts get-session-token --token-code 123456.

  • First arg: the shared secret. Either the base32 seed from your 2FA setup, or a full otpauth://totp/... URI. Spaces, hyphens, lowercase, and padding in a base32 seed are all fine.
  • digits=N option: code length, 6 to 10 (default: 6)
  • period=N option: how long each code is valid, in seconds (default: 30). A duration string like "60s" also works.
  • algorithm=S option: SHA1 (default), SHA256, or SHA512
  • encoding=S option: how the secret itself is encoded, base32 (default), hex, or ascii

When the secret is an otpauth:// URI, any digits, period, and algorithm params in the URI are used, and explicitly passed options override them.

The seed is a long-lived credential, so keep it @internal (resolved by varlock, never injected into your app or child processes) and @sensitive. Only the generated code gets injected.

# TOTP seed from your provider's 2FA setup, encrypted at rest
# @internal @sensitive
MFA_SECRET=varlock(local:abc123...)

# @sensitive
MFA_CODE=generateOtp($MFA_SECRET)

# non-default params
# @sensitive
LEGACY_CODE=generateOtp($LEGACY_SEED, digits=8, period=60, algorithm=SHA256)

Then hand the code to whatever needs it. Tools that take a flag need the expansion to happen in the child shell, so use single quotes:

varlock run -- sh -c 'aws sts get-session-token --serial-number $AWS_MFA_ARN --token-code $MFA_CODE'

Tools that read a code from the environment need no extra wrapping, since varlock injects it directly.

To get the seed in the first place, you generally have to enroll (or re-enroll) the second factor: on the "scan this QR code" screen, take the manual-entry / setup-key option and save that string. Most authenticator apps will not show you a seed after the fact.

:::note[One code per command] A code covers one authenticated call, and providers generally reject a reused one. If a command makes several authenticated calls in a run, prefer handing it the seed and letting it mint a code per call. fledgling, for example, takes FLEDGLING_OTP_SECRET. :::

:::caution[Codes cannot be cached] Wrapping this in cache() is an error, since a cached code is expired by definition. Cache the secret instead: generateOtp(cache(op("op://vault/aws/mfa seed"))). :::

A few other things to know:

  • Codes expire. They rotate every period seconds, so generate at the moment of use. A long command that only needs the code at the very end can outlive it.
  • Your clock has to be right. Codes are derived from the current time. Check clock sync before assuming the seed is wrong.
  • This weakens your second factor. A seed sitting next to the token it protects, on the same machine, is no longer an independent factor. That can be a reasonable trade for a local workflow where the seed is encrypted at rest and never injected into your app. It is a bad trade in CI, where OIDC workload identity or credentials that do not require 2FA are the better answer.

OAuth tokens

### `oauth()`

Exchanges a long-lived OAuth credential for a short-lived access token by calling the provider's token endpoint. The item resolves to a fresh access token, and only that token is injected. The refresh token and client secret stay in your vault, referenced as @internal items that never reach your app or child processes. See the OAuth guide for the full workflow.

Tokens are cached (encrypted, according to your cache mode) and reused until the provider-reported expiry, so repeated invocations do not hit the token endpoint. When a provider rotates refresh tokens on each use (Google and Slack do), the rotated token is stored in the cache and used for the next refresh automatically; the configured refresh token is just the bootstrap.

An optional first positional arg references an @oauthProvider instance by id, which supplies tokenUrl, clientId, clientSecret, and clientAuth so several items can share one client config. Item-level args override provider-level ones.

Options:

  • tokenUrl=S: the provider's token endpoint (required unless a provider instance or service account key supplies it). Must be https (plain http is allowed for localhost).
  • grant=S option: refresh_token (default), client_credentials, or jwt_bearer
  • refreshToken=R: the refresh token, usually a reference to another item. Required for the refresh_token grant unless a provider instance is referenced, in which case omitting it means "use the login-provisioned token" (see below).
  • clientId=R: the OAuth client id (required unless a provider instance supplies it; optional for jwt_bearer)
  • clientSecret=R option: the OAuth client secret. Not needed for public (PKCE) clients.
  • clientAuth=S option: how client credentials are sent, body (default) or basic for HTTP basic auth. Some providers (e.g. Notion) require basic.
  • scopes=R option: a space-delimited string or an array of scope strings
  • params={...} option: extra form params for the token request, e.g. params={ audience="..." } for Auth0
  • skew=N option: refresh this long before the reported expiry, in seconds or a duration string (default: 60s)

jwt_bearer-only options (RFC 7523, e.g. Google service accounts): instead of a stored credential, varlock signs a short-lived RS256 assertion with a private key and exchanges it for an access token.

  • serviceAccountKey=R: a Google-style service account key JSON (supplies the signing key, issuer, and token endpoint)
  • privateKey=R + issuer=R: raw PEM key and iss claim, for non-Google providers
  • subject=R option: sub claim, for providers that support impersonation (e.g. Google domain-wide delegation)
  • audience=S option: aud claim override (defaults to the token endpoint)
# @oauthProvider(id=google, preset=google, clientId=$GOOGLE_CLIENT_ID, clientSecret=$GOOGLE_CLIENT_SECRET)
# ---
# @internal
GOOGLE_CLIENT_ID=1234-abcd.apps.googleusercontent.com
# @internal @sensitive
GOOGLE_CLIENT_SECRET=op("op://dev/google-oauth/client secret")

# no refreshToken: provision once with `varlock oauth login google`
# @sensitive
DRIVE_TOKEN=oauth(google, scopes="https://www.googleapis.com/auth/drive.readonly")

# or pass a vault-stored refresh token explicitly (e.g. for CI)
# @internal @sensitive
GOOGLE_REFRESH_TOKEN=op("op://dev/google-oauth/refresh token")
# @sensitive
SHEETS_TOKEN=oauth(google, refreshToken=$GOOGLE_REFRESH_TOKEN, scopes="https://www.googleapis.com/auth/spreadsheets.readonly")

# fully inline, no provider instance
# @sensitive
API_TOKEN=oauth(tokenUrl="https://myorg.auth0.com/oauth/token", grant="client_credentials", clientId=$AUTH0_CLIENT_ID, clientSecret=$AUTH0_CLIENT_SECRET, params={ audience="https://api.myorg.com" })

# Google service account (jwt_bearer): the key file supplies everything
# @internal @sensitive
GCP_SA_KEY=op("op://infra/gcp-sa/key json")
# @sensitive
GCP_TOKEN=oauth(grant="jwt_bearer", serviceAccountKey=$GCP_SA_KEY, scopes="https://www.googleapis.com/auth/cloud-platform")

A few things to know:

  • The initial refresh token has to come from somewhere. Either run varlock oauth login once (stores it in the encrypted cache, per machine), or run your provider's authorization flow elsewhere and store the token in your vault, passing it via refreshToken. The explicit form is the right one for CI.
  • Login-provisioned tokens are shared per provider. Items referencing the same provider without their own refreshToken share one refresh token; each item still gets its own access token scoped to its scopes.
  • Rotation needs a persistent cache. With caching disabled (--skip-cache, or no cache store available), a provider that rotates refresh tokens will invalidate the configured one after the first exchange. varlock prints a warning when this happens.
  • Concurrent invocations share one refresh. Parallel varlock run processes on the same machine coordinate through a lock, so a rotating provider sees one exchange, not a stampede.
  • If a refresh fails with invalid_grant, the refresh token is expired or revoked. Re-run varlock oauth login (or re-provision the vault-stored token).

:::caution[Do not wrap in cache()] Wrapping this in cache() is an error. oauth() already caches tokens according to their provider-reported expiry; a generic cache TTL would serve expired tokens. Wrapping the inputs in cache() is fine. :::

Caching

### `cache()`

Wraps any resolver to cache its result. See caching guide for more details.

  • First arg: the resolver to cache
  • ttl=D option: how long to cache (default: forever). Accepts duration strings with ms, s, m, h, d, w suffixes (long forms and plurals also work), bare numbers as milliseconds, or the keyword forever to cache until manually cleared.
  • key=S option: use an explicit cache key instead of the auto-generated one. Useful when the same cached value should be shared across files or when you want a stable key that doesn't change with resolver edits.
    • Keys must be non-empty printable text (no control characters) and at most 2048 characters.

The cache automatically invalidates when you change the wrapped resolver expression (unless using a custom key).

# Cache a random UUID forever (until manually cleared)
INSTANCE_ID=cache(randomUuid())

# Cache an API token for 1 hour
AUTH_TOKEN=cache(exec(`get-token.sh`), ttl="1h")

# Cache for 30 minutes
TEMP_KEY=cache(randomHex(32), ttl="30m")

# Use an explicit cache key (shared across files/projects)
SHARED_TOKEN=cache(exec(`fetch-org-token.sh`), ttl="1d", key="org-auth-token")

Use the varlock cache CLI command to view or clear the disk cache.

Use --clear-cache or --skip-cache flags on varlock load / varlock run / varlock printenv to control caching behavior for a single invocation.

Global cache behavior is configured with the @cache root decorator.

In memory mode, varlock cache will not show in-memory entries from a running process.

For strategy recommendations and troubleshooting, see the Caching guide.

:::tip Plugin authors can also use the cache API via plugin.cache.getOrSet() (or get() / set()) to cache expensive API calls. See the Plugins guide for more information. :::

Encryption

### `varlock()`

Decrypts a locally encrypted value, or prompts for a new secret to encrypt. This is the built-in resolver for varlock's device-local encryption feature.

Decrypt mode: pass an encrypted payload to decrypt at load time:

# @sensitive
API_KEY=varlock("local:<encrypted-payload>")

Prompt mode: prompts the user to enter a secret, encrypts it, and writes the encrypted value back to the source file:

# @sensitive
API_KEY=varlock(prompt)
# also valid as a key=value param:
API_KEY=varlock(prompt=1)

On first run with prompt mode, you'll be asked to enter the secret value. Once entered, the file is automatically updated with the encrypted payload. On macOS with Secure Enclave, a native dialog with biometric authentication is used.

Values are encrypted using the best available backend on your platform. See the Local encryption guide for details.

Encrypted payload lifecycle:

  • Store encrypted payloads (varlock("local:...")) in local override files (typically .env.local)
  • Decryption happens at runtime during varlock load / varlock run
  • Use varlock reveal when you need to inspect a decrypted value interactively
### `keychain()`

Reads a secret from the macOS Keychain. This built-in resolver communicates through Varlock's native Swift daemon, enforcing biometric (Touch ID) authentication and per-session access control. See the macOS Keychain page for full documentation.

Array args:

  • service (optional): Service name of the keychain item (positional shorthand)
  • prompt (optional): Enter interactive picker mode

Key/value args:

  • service (optional): Service name of the keychain item
  • account (optional): Account identifier for the keychain item
  • keychain (optional): Name of a specific keychain to search (e.g., "System")
  • field (optional): Specific field to extract from the keychain item
  • prompt (optional): If set, opens a native picker dialog for interactive selection
# Positional shorthand
DATABASE_PASSWORD=keychain("com.company.database")

# Named service with account
ADMIN_PW=keychain("com.company.db", account="admin")

# Interactive picker mode, writes back resolved reference
NEW_SECRET=keychain(prompt)

:::caution keychain() is only available on macOS. For cross-platform local encryption, use varlock() instead. :::