JSON schema conversion benchmarks

[EskiMojo14]

see #186 - opening separately as I don't have permission to push to the fork branch.

Credit has been added to the original commit.

[netlify]

✅ Deploy Preview for schemabenchmarks ready!

Name	Link
🔨 Latest commit	a044722
🔍 Latest deploy log	https://app.netlify.com/projects/schemabenchmarks/deploys/6a73bacf457b5e00089b67f3
😎 Deploy Preview	https://deploy-preview-196--schemabenchmarks.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

Extends the benchmark suite and website UI to include a new “JSON Schema Conversion” benchmark page, along with the underlying schemas/types, bench orchestration, and CI plumbing to generate and publish json-schema.json.

Changes:

• Add JSON Schema conversion benchmark definitions (targets/directions/support matrix), generation tests, and accepted-output fixtures.
• Add a new /json-schema section in the website (route, content, plots/tables/cards, sidebar navigation, offline asset inclusion).
• Add bench scripts + workflows to produce and collect bench/json-schema.json, and wire it into bench:all.

Reviewed changes

Copilot reviewed 72 out of 75 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
website/vite.config.ts	Adds JSON Schema filter/icon labels to the preloaded icon set.
website/src/shared/components/sidebar/index.tsx	Adds optional group subheaders with ARIA labeling.
website/src/shared/components/sidebar/groups.ts	Adds a “JSON Schema” sidebar group + subheader support.
website/src/shared/components/plot/index.stories.tsx	Simplifies plot story render wrapper.
website/src/shared/components/code/index.stories.tsx	Switches story import to barrel (.).
website/src/shared/components/admonition/index.stories.tsx	Switches story import to barrel (.).
website/src/routeTree.gen.ts	Adds generated routes for /json-schema and /json-schema/conversion.
website/src/routes/_home/content.mdx	Adds homepage section + plots linking to JSON Schema Conversion.
website/src/routes/_benchmarks/stack/-components/table/index.tsx	Removes outer card wrapper; standardizes table transition name.
website/src/routes/_benchmarks/stack/-components/card/index.stories.tsx	Story import cleanup + CSS import ordering.
website/src/routes/_benchmarks/json-schema/styles.css	Adds shared JSON Schema page styles and imports component CSS.
website/src/routes/_benchmarks/json-schema/route.tsx	Introduces JSON Schema route layout + stylesheet injection.
website/src/routes/_benchmarks/json-schema/index.tsx	Redirects /json-schema/ to /json-schema/conversion.
website/src/routes/_benchmarks/json-schema/conversion/index.tsx	Adds conversion page route (filters, loader prefetch, metadata).
website/src/routes/_benchmarks/json-schema/conversion/content.mdx	Adds page explanation + methodology notes.
website/src/routes/_benchmarks/json-schema/conversion/-query.ts	Adds query for json-schema.json (client fetch + server import).
website/src/routes/_benchmarks/json-schema/conversion/-constants.ts	Adds filter chip props + optional search schemas.
website/src/routes/_benchmarks/json-schema/conversion/-components/table/index.tsx	Adds JSON Schema results table with compare UI + schema viewer.
website/src/routes/_benchmarks/json-schema/conversion/-components/table/index.stories.tsx	Adds Storybook story for the JSON Schema table.
website/src/routes/_benchmarks/json-schema/conversion/-components/table/index.css	Adds table-specific CSS helpers.
website/src/routes/_benchmarks/json-schema/conversion/-components/results.tsx	Adds responsive results renderer (table vs cards).
website/src/routes/_benchmarks/json-schema/conversion/-components/plot/index.tsx	Adds plot component for JSON Schema conversion results.
website/src/routes/_benchmarks/json-schema/conversion/-components/plot/index.stories.tsx	Adds Storybook story for JSON Schema plot.
website/src/routes/_benchmarks/json-schema/conversion/-components/matrix.tsx	Adds support matrix (per-target/direction unsupported reasons).
website/src/routes/_benchmarks/json-schema/conversion/-components/json-schema.tsx	Adds UI to display generated JSON schema in a tooltip/code block.
website/src/routes/_benchmarks/json-schema/conversion/-components/card/index.tsx	Adds card view for JSON Schema results (mobile layout).
website/src/routes/_benchmarks/json-schema/conversion/-components/card/index.stories.tsx	Adds Storybook story for JSON Schema card.
website/src/routes/_benchmarks/json-schema/conversion/-components/card/index.css	Adds styling for JSON Schema result cards.
website/src/routes/_benchmarks/download/-components/table/index.tsx	Removes outer card wrapper; standardizes transition name.
website/src/routes/_benchmarks/_runtime/codec/-components/table/index.tsx	Removes outer card wrapper; standardizes transition name.
website/src/routes/_benchmarks/_runtime/-hooks.ts	Generalizes sorting helpers to runtime + JSON schema result types.
website/src/routes/_benchmarks/_runtime/-components/table/index.tsx	Refactors to RuntimeResult, adds formatDuration, improves compare for 0ms rows, removes wrapper.
website/src/routes/_benchmarks/_runtime/-components/results.tsx	Refactors to RuntimeResult.
website/src/routes/_benchmarks/_runtime/-components/plot/index.tsx	Refactors to RuntimeResult + uses formatDuration; factors selection logic.
website/src/routes/_benchmarks/_runtime/-components/card/index.tsx	Uses formatDuration; refactors to RuntimeResult.
website/package.json	Copies bench/json-schema.json into website public assets.
website/offline/opts.ts	Includes json-schema.json in offline assets.
website/e2e/nav.test.ts	Adds nav smoke check for “JSON Schema” link.
website/.gitignore	Ignores public/json-schema.json.
utils/src/index.ts	Adds formatDuration to avoid empty output for zero durations.
schemas/vite.config.ts	Adds Vitest typecheck project for .test-d.ts tests.
schemas/test/types.test-d.ts	Adds type-level assertions for JSON schema APIs/config typing.
schemas/test/libraries.node.test.ts	Adds runtime JSON Schema generation validation + accepted-shape assertions.
schemas/test/accepted-json-schemas.ts	Adds allowlisted generated schemas per target/direction.
schemas/src/types.ts	Introduces JSON Schema benchmark types/targets/directions + assertions; adds jsonSchema to config.
schemas/src/data.ts	Adds benchmark subject input/output sample data for JSON schema validation.
schemas/scripts/gen-accepted-json-schemas.ts	Adds generator script to refresh accepted schema fixtures.
schemas/package.json	Adds script + deps for JSON schema generation (valibot/joi/zod adapters).
schemas/libraries/zod/v3/benchmarks.ts	Adds JSON schema generation via zod-to-json-schema.
schemas/libraries/zod/mini/benchmarks.ts	Adds JSON schema generation via z.toJSONSchema.
schemas/libraries/zod/benchmarks.ts	Adds JSON schema generation + standard JSON schema support metadata.
schemas/libraries/valibot/benchmarks.ts	Adds JSON schema generation via @valibot/to-json-schema + standard interface wrapper.
schemas/libraries/typia/benchmarks.ts	Adds precompiled JSON schema/OpenAPI generation case.
schemas/libraries/typebox/benchmarks.ts	Adds “already JSON schema” case for input direction.
schemas/libraries/sury/benchmarks.ts	Adds JSON schema generation and standard JSON schema opt-in support.
schemas/libraries/joi/benchmarks.ts	Adds JSON schema generation via joi-to-json.
schemas/libraries/effect/benchmarks.ts	Adds JSON schema generation via effect/JSONSchema.
schemas/libraries/effect/@beta/benchmarks.ts	Adds JSON schema generation (draft 2020-12 input only).
schemas/libraries/ata-validator/benchmarks.ts	Adds “already JSON schema” case using ata-validator/t.
schemas/libraries/arktype/benchmarks.ts	Adds JSON schema generation + standard JSON schema support metadata.
schemas/libraries/ajv/benchmarks.ts	Adds “already JSON schema” case using typed JSONSchemaType.
pnpm-lock.yaml	Locks new dependencies (@valibot/to-json-schema, joi-to-json, zod-to-json-schema).
package.json	Adds bench:json-schema and wires it into bench:all.
bench/src/scripts/bench/library.ts	Refactor to read optimizeType directly from registry entries.
bench/src/scripts/bench/json-schema/orchestrate.ts	Adds orchestrator to run json-schema bench per library and merge outputs.
bench/src/scripts/bench/json-schema/library.ts	Adds per-library JSON schema bench runner + support reporting.
bench/src/results/types.ts	Adds JSON schema result/support schemas; splits runtime vs base bench schema types.
bench/src/bench/registry.ts	Extends registry typing to include JSON schema bench cases.
bench/package.json	Exposes json-schema.json export + adds bench:json-schema script.
bench/json-schema.json	Adds generated JSON schema benchmark results + support matrix data.
bench/json-schema.d.json.ts	Adds typed JSON module declaration for json-schema.json.
.github/workflows/bench.yml	Adds json-schema bench job and collects its artifact.
.github/workflows/bench-pr.yml	Adds json-schema bench job for PRs and collects its artifact.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[DZakh]

Looks good. Thanks for picking it up. I think the only comment that support matrix on the home page would be more relevant and interesting to see than performance of draft-20.

Also, the order of home content doesn't follow the sidebar, which I found misleading.

[Copilot]

Pull request overview

Copilot reviewed 80 out of 83 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

Comments suppressed due to low confidence (4)

website/src/routes/_benchmarks/json-schema/conversion/-components/table/index.tsx:164

• The “Standard JSON Schema” column renders an empty cell when a library has no support (or when the result isn’t runtime), but the page copy describes a “None” state. This makes the table harder to interpret and diverges from the documented values.
  website/src/routes/_home/index.tsx:5
• This is the only #src/... import in the website that includes a .ts extension. Elsewhere (including routes/_benchmarks/json-schema/conversion/index.tsx:22) the same module is imported without an extension; keeping this consistent avoids resolution differences between tooling.
  website/src/routes/_benchmarks/json-schema/conversion/-components/matrix.tsx:97
• The support matrix leaves the “Standard JSON Schema” cell blank when standardJsonSchema is missing, even though the UI copy enumerates a “None” state. Consider rendering an explicit "None" value so the matrix is unambiguous.
  website/src/routes/_benchmarks/json-schema/conversion/-components/matrix.tsx:14
• This import includes a .ts extension, but other imports of the same module in this feature use extensionless paths (e.g. conversion/index.tsx). For consistency and to avoid toolchain-specific resolution issues, prefer omitting the extension.

[sinclairzx81]

@EskiMojo14 Hello!

AJV and ATA currently show as supporting both 2020-12 and Draft-7 ... but TB is only showing 2020-12

https://deploy-preview-196--schemabenchmarks.netlify.app/

---

Reference:

Here is the specification coverage implemented by TB:

https://github.com/sinclairzx81/typebox#coverage

Here is TB type-level support for Draft 7 and Draft 2020-12:

TypeScript Reference Here

---

Question:

Does it make sense to include AJV, ATA, and TB in a benchmark specific to JSON Schema conversion? It isn't really an apples-to-apples comparison, and they appear to be visually penalized for not having JSON Schema conversion functions, even though they all intrinsically support the specification, which seems a bit misleading.

Would it not be better to narrow participation to libraries that actually provide X > JSON Schema conversion?

[DZakh]

In my understanding, it makes sense to include all libraries, because these are 2 different operations:

• Create a schema/validator from JSON Schema - this is where TB, AJV, ATA have good coverage
• Create JSON Schema or Standard JSON Schema from schema - this is where TB and others are penalized, because they use JSON Schema for internal implementation, which doesn't convert to all standards out of the box.

Both of them are important and have different use cases. So I think ideally there should be two separate benchmarks From JSON Schema and To JSON Schema.

I also think that the PR, as it is, is already good, and this should go as a follow-up PR.

[DZakh]

I also think that the JSON Schema coverage number and TypeScript inference support are good cases for the follow-up PR.

[DZakh]

To summarize, to address the confusion @sinclairzx81 highlighted, I'd rename "conversion" to "To JSON Schema".

And then add a separate PR for "From JSON Schema" because this is a separate thing. I can also help create a foundation for it.

[EskiMojo14]

for JSON schema based libraries, we already measure "from JSON schema" - that's what the initialization benchmark is. there are a few libraries that support creating schemas from JSON without being based on it, which might be interesting to benchmark.

I can see the argument that it doesn't make sense to measure "conversion" for libraries that use JSON schema as their base. I think the cleanest solution is to remove them from this - the compliance benchmark (#198) will be a more useful "support matrix" than this provides, anyway.

I'm not entirely sure what the best approach for Typia in particular would be - they don't use JSON schema as a base, but the "conversion" is still in build time.

[EskiMojo14]

okay, lots of changes later, here's a summary:

• renamed existing benchmarks to conversion.toJson, and removed non-runtime conversions.
• added conversion.fromJson benchmarks (maybe missing some)
• support matrix removed in favour of compliance testing (will merge both together)

thoughts appreciated :)

[Copilot]

Pull request overview

Copilot reviewed 86 out of 89 changed files in this pull request and generated 1 comment.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

Suppressed comments (4)

website/e2e/nav.test.ts:28

• Sidebar link text is "JSON to Schema" (JSON uppercase) in the UI; this test currently uses "Json to Schema", which will fail to find the link by accessible name.
  website/src/routes/_benchmarks/json-schema/_conversion/to-json/index.tsx:49
• openGraph.url points to "/json-schema/_conversion/to-json", but the public route is "/json-schema/to-json" (and from-json already uses the public path). This will generate incorrect OG/canonical metadata.
  website/e2e/nav.test.ts:27
• Sidebar link text is "Schema to JSON" (JSON uppercase) in the UI; this test currently uses "Schema to Json", which will fail to find the link by accessible name.

This issue also appears on line 28 of the same file.
website/src/routes/_home/index.tsx:5

• This is the only #src/... import in the website codebase that includes a .ts extension (the other imports in this file omit extensions). Keeping it consistent avoids mixed-resolution behavior across tooling (TS, Vite, eslint/oxlint).

[DZakh]

Looks good! I like the split.

1. Still think that performance is not really relevant for the conversion cases and should be secondary, because it's usually called only once at process startup. But it's not critical at all.
2. Compliance testing is good, but it makes toJSONSchema dependent on the library's fromJSONSchema ability. I think it'd be better to have a simple support matrix for toJSONSchema, including Standard Schema support, and then a separate compliance matrix + roundtrip for the fromJSONSchema case.

Nothing blocking, just my opinion.

[DZakh]

Although, as you said, the support matrix can be merged into compliance, just for libraries not supporting fromJSONSchema, it'll be an unknown JSON Schema compatibility number. Need to experiment whether 1 or 2 tables will look better/cleaner.

[EskiMojo14]

that's a fair point - the compliance testing is a support matrix for from JSON, but not for to JSON. I'll have a think on how best to present that information - like you said, having two tables made the page feel overcrowded.

[EskiMojo14]

okay - added support matrix back, and added tabs to the page so only one table is shown at a given time. i also created a card version of the support matrix for mobile users.

[DZakh]

Hi, I like how the UI looks. I just have a few comments regarding data; I hope I don't overwhelm you 😅

• ajv and ata-validators don't make sense for to json schema matrix - they are not schemas, but validators. You create a JSON schema from a validator, but you can't create a JSON schema from a validator.
• From the other side I think TypeBox should be included, since it's not a validator, but pretty much a full-fledged schema - the fact that it uses JSON Schema as internal representation is an implementation detail. But I believe it should only be in the Draft7 section, and the other ones should be marked as +- or something like this, since there's no automatic conversion, only if the user specifies it during schema creation.
• I think it would be nice to have an extra column for toJSONSchema support matrix showing Standard JSON Schema support with native|native-opt-in|additional-package options - the table looks like a nice place for it
• Sury supports S.fromJSONSchema 😁

[DZakh]

Ah, and forgot to mention one more. I think the matrix on the home page would be a better overview