-
Notifications
You must be signed in to change notification settings - Fork 6
Coercion through preprocessing PoC #465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 13 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
c243856
feat: add specification, design, and tasks for automatic environment …
yamcodes 31d9316
docs: Refactor design document to detail the decision and rationale f…
yamcodes 2777437
feat: Implement automatic type coercion for numbers and booleans in r…
yamcodes d9574c0
[autofix.ci] apply automated fixes
autofix-ci[bot] f88d25c
Merge branch 'main' into 228-coercion-stringnumber-stringboolean-etc
yamcodes 4aecd75
Merge branch 'main' into 228-coercion-stringnumber-stringboolean-etc
yamcodes 3aa3f6c
Merge branch 'main' into 228-coercion-stringnumber-stringboolean-etc
yamcodes da67f34
[autofix.ci] apply automated fixes
autofix-ci[bot] a5ac144
refactor: format
yamcodes 5a90cd2
Merge branch '228-coercion-stringnumber-stringboolean-etc' of https:/…
yamcodes 53cc46d
build: Disable sourcemap generation for Bun build.
yamcodes 3afa7fa
feat: Add custom type transformation example and remove sourcemap fro…
yamcodes 2e79e95
chore: update dependencies
yamcodes a890b26
chore: update package-lock.json to reflect dependency changes
yamcodes 197f501
Merge branch 'main' into 228-coercion-stringnumber-stringboolean-etc
yamcodes 50c91b4
fix package lock
yamcodes 1551753
Merge branch 'main' into 228-coercion-stringnumber-stringboolean-etc
yamcodes File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # Design: Coercion | ||
|
|
||
| ## Architecture | ||
| The coercion logic will be implemented as a preprocessing step within the `createEnv` function. | ||
|
|
||
| ### Flow | ||
| 1. **Input**: `createEnv` receives a schema definition (`def`) and an environment object (`env`). | ||
| 2. **Inspection**: We inspect `def` to identify keys that expect primitive types (number, boolean) but will receive strings from `env`. | ||
| * This inspection primarily targets schema definitions provided as plain objects with string values (e.g., `{ PORT: "number" }`). | ||
| * Complex ArkType definitions (already compiled types) may be skipped or require advanced introspection (out of scope for initial implementation). | ||
| 3. **Coercion**: | ||
| * For each identified key, we check the corresponding value in `env`. | ||
| * If the target type is `number` (or subtypes like `number.port`, `number.epoch`), we attempt to convert the string to a number using `Number()` or `parseFloat()`. | ||
| * If the target type is `boolean`, we convert "true" to `true` and "false" to `false`. | ||
| 4. **Validation**: The modified `env` object (with coerced values) is passed to the ArkType schema for validation. | ||
|
|
||
| ## Decisions | ||
|
|
||
| ### Decision: Use Preprocessing for Coercion | ||
| We decided to implement coercion as a preprocessing step that runs *before* ArkType validation, rather than using ArkType's native "morphs" or scope-level overrides. | ||
|
|
||
| **Rationale:** | ||
| 1. **Scope Limitations**: As confirmed by the ArkType creator, there is no mechanism to apply a morph to an entire scope (e.g., "all numbers"). We would have to manually override `number` and every subtype (`number.port`, `number.epoch`, etc.), which is brittle and unscalable. | ||
|
yamcodes marked this conversation as resolved.
|
||
| 2. **Separation of Concerns**: Coercion (parsing a string into a primitive) is distinct from Validation (checking if that primitive meets criteria). Keeping coercion separate allows `arkenv` to handle the "environment variable boundary" explicitly, ensuring that `number` in the schema always validates a real JavaScript number. | ||
| 3. **Complexity**: Implementing type-level mapping for global coercion would introduce significant complexity to the types, whereas a runtime preprocessor is straightforward and easier to maintain. | ||
|
|
||
| **Alternatives Considered:** | ||
| * **ArkType Morphs**: We considered using `type("string").pipe(...)` or overriding keywords in the scope. This was rejected because it requires manual per-type configuration or complex scope manipulation that doesn't propagate to sub-keywords. | ||
| * **Manual Parsing**: Continuing with the current state where users manually pipe string types. This was rejected as it degrades developer experience. | ||
|
|
||
| ## Risks / Trade-offs | ||
| * **String Definitions**: This approach relies on inspecting the schema definition. It works best when users provide string definitions (e.g., `{ PORT: "number" }`). If a user provides a pre-compiled `type("number")`, we cannot easily inspect it to apply coercion, meaning those values might remain strings and fail validation. We will document this limitation. | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| ### `coerce` Utility | ||
| We will create a utility function `coerce(def: Record<string, unknown>, env: Record<string, string | undefined>)` that returns a new environment object. | ||
|
|
||
| ```typescript | ||
| function coerce(def: Record<string, unknown>, env: Record<string, string | undefined>) { | ||
| const coerced = { ...env }; | ||
| for (const key in def) { | ||
| const typeDef = def[key]; | ||
| if (typeof typeDef === "string") { | ||
| if (typeDef.startsWith("number")) { | ||
| // Coerce to number | ||
| } else if (typeDef === "boolean") { | ||
| // Coerce to boolean | ||
| } | ||
| } | ||
| } | ||
| return coerced; | ||
| } | ||
| ``` | ||
|
|
||
| ### Integration | ||
| In `createEnv`: | ||
|
|
||
| ```typescript | ||
| export function createEnv(def, env = process.env) { | ||
| // ... | ||
| const coercedEnv = isPlainObject(def) ? coerce(def, env) : env; | ||
| const validatedEnv = schema(coercedEnv); | ||
| // ... | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # Coercion | ||
|
|
||
| ## Problem | ||
| Environment variables are always strings at runtime, but users want to treat them as typed primitives without manual conversion. | ||
|
|
||
| **Current state:** | ||
| ```typescript | ||
| // Manual conversion required | ||
| const env = arkenv({ | ||
| PORT: type("string").pipe(str => Number.parseInt(str, 10)), | ||
| DEBUG: type("string").pipe(str => str === "true") | ||
| }); | ||
| ``` | ||
|
|
||
| **Desired state:** | ||
| ```typescript | ||
| // Coercion | ||
| const env = arkenv({ | ||
| PORT: "number", // "3000" → 3000 | ||
| DEBUG: "boolean", // "true" → true | ||
| TIMESTAMP: "number.epoch" // "1640995200000" → 1640995200000 | ||
|
yamcodes marked this conversation as resolved.
|
||
| }); | ||
| ``` | ||
|
|
||
| ## Solution | ||
| Implement an automatic coercion layer in `arkenv` that runs before ArkType validation. This layer will inspect the provided schema definition and, where possible, convert string environment variables into their target primitive types (number, boolean) so that ArkType can validate them as such. | ||
|
|
||
| This approach allows `arkenv` to support "native" feeling environment variables while leveraging ArkType's powerful validation for the final values. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Spec: Coercion | ||
|
|
||
| ## ADDED Requirements | ||
|
|
||
| ### Requirement: Coerce numeric strings to numbers | ||
| The system MUST coerce environment variable strings to numbers when the schema definition specifies `number` or a `number.*` subtype. | ||
|
|
||
| #### Scenario: Basic number coercion | ||
| Given a schema `{ PORT: "number" }` | ||
| And an environment `{ PORT: "3000" }` | ||
| When `arkenv` parses the environment | ||
| Then the result should contain `PORT` as the number `3000` | ||
|
|
||
| #### Scenario: Number subtype coercion | ||
| Given a schema `{ TIMESTAMP: "number.epoch" }` | ||
| And an environment `{ TIMESTAMP: "1640995200000" }` | ||
| When `arkenv` parses the environment | ||
| Then the result should contain `TIMESTAMP` as the number `1640995200000` | ||
|
yamcodes marked this conversation as resolved.
|
||
|
|
||
| ### Requirement: Coerce boolean strings to booleans | ||
| The system MUST coerce environment variable strings "true" and "false" to boolean values when the schema definition specifies `boolean`. | ||
|
|
||
| #### Scenario: Boolean true coercion | ||
| Given a schema `{ DEBUG: "boolean" }` | ||
| And an environment `{ DEBUG: "true" }` | ||
| When `arkenv` parses the environment | ||
| Then the result should contain `DEBUG` as the boolean `true` | ||
|
|
||
| #### Scenario: Boolean false coercion | ||
| Given a schema `{ DEBUG: "boolean" }` | ||
| And an environment `{ DEBUG: "false" }` | ||
| When `arkenv` parses the environment | ||
| Then the result should contain `DEBUG` as the boolean `false` | ||
|
|
||
| ### Requirement: Pass through non-coercible values | ||
| The system MUST pass through values unchanged if they do not match a coercible type definition or if coercion fails (letting ArkType handle the validation error). | ||
|
|
||
| #### Scenario: String pass-through | ||
| Given a schema `{ API_KEY: "string" }` | ||
| And an environment `{ API_KEY: "12345" }` | ||
| When `arkenv` parses the environment | ||
| Then the result should contain `API_KEY` as the string `"12345"` | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 90
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 41
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 794
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 2431
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 643
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 752
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 41
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 41
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 671
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 2613
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 654
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 874
🏁 Script executed:
Repository: yamcodes/arkenv
Length of output: 41
Use the built-in
"number"type instead of customNumber()conversion.Replace the manual
.pipe((value) => Number(value))with ArkType's built-in"number"type, which handles string-to-number coercion safely with proper NaN validation:The built-in
"number"type already coerces valid numeric strings (e.g.,"3000"→3000) and rejects invalid ones (e.g.,"abc") during validation. This follows the guideline to leverage ArkType's built-in types where possible and keeps the schema readable and consistent with other fields likePORT: "number.port".📝 Committable suggestion
🤖 Prompt for AI Agents