[RFC] Validation (Zod, Standard Schema etc)

[franky47]

Draft PR: #448.

There's been a few requests for using Zod to validate parsed search params, both on the server and on the client.

This makes a lot of sense since:

1. Only basic validation is currently provided in the built-in parsers
2. Implementing custom parsers just to check that a number is in a particular range is not a great DX
3. Zod is popular and likely to already be present in the stack

A proposition is to add a new method to the parsers builder API, withValidation, that takes in a Zod schema .parse function:

const emailParser = parseAsString.withValidation(z.string().email().parse)
const posIntParser = parseAsInteger.withValidation(z.number().int().positive().parse)
// Works with compound parsers:
const arrayParser = parseAsArrayOf(parseAsString)
  .withValidation(z.array(z.union([z.literal('foo'), z.literal('bar')])).parse)
// or validate on each item:
const arrayParser = parseAsArrayOf(parseAsString
  .withValidation(z.union([z.literal('foo'), z.literal('bar')]).parse)
)

Since errors thrown in parsers result in null or the default value (if specified) being returned, this should work and would not actually introduce a peer dependency on Zod. It would accept any function that takes in an unknown argument and returns the parser output type. Technically we'd only call the validator if the query value has successfully been parsed / deserialized from a string, so T would also be a suitable input type for the validator.

Why not use Zod directly in place of parsers ?

Zod can behave like the parse method: take a string, hydrate it into a JS data type, and run some validation on top while doing so.

However, it does not provide the counterpart serialization function to transform that JS data type back into a string, which is necessary on query state updates.

Also, it would require all Zod schema definitions to be rooted in z.string().transform(s => whatever), which is not what existing application logic would define (schema definitions are likely to be reused or derived from domain-specific definitions).

Caveats & Limitations

Validation needs to happen synchronously. This is because the parsing + validation is done in each hook's internal state initialisation, (getting the correct value from the URL on mount), which must return a value synchronously.

Questions

1. Should the validation run on writes as well as reads ? Ie: should this fail:

const [positive, setPositive] = useQueryState(
  'pos',
  parseAsInteger.withValidation(z.number().positive().parse)
)

setPositive(-1) // Should this fail?

2. Should multiple calls to withValidation compose validations or reset them? Eg:

const dice = parseAsInteger
  .withValidation(z.number().gte(1).parse)
  .withValidation(z.number().lte(6).parse)

// What should be the result for trying to parse "0" here?
// - if compose validations, then `null`
// - if overriden, then `0`

Ecosystem

While Zod is popular, an emerging standard schema has been developed and is being adopted by schema-consuming tools. Conforming to its specs would allow using Zod or any other standard-schema compliant validation library.

[CogitoBurrito]

Is it possible to make this feature support multiple validators via TypeSchema?

[franky47]

I was going to say yes, by currying the schema into the assert function, but I just noticed that the output is a Promise, and validation needs to run synchronously here.

Edit: I added a caveat section to emphasize this, thanks for pointing it out!

[CogitoBurrito]

The reason I propose this is that soon people will be asking, 'Can you support X lib as the validator?

We can consider this options again once the author adds synchronous validation.

[franky47]

Some of the underlying validation libraries use async validation by default (Yup being the most popular one). Not much we can do there.

For any other synchronous APIs, as long as it conforms to the validator: (input: unknown) => T type, and throws on invalid inputs, it should work just fine.

[franky47]

I added some questions that are currently left open for discussion, and a draft PR.

[CogitoBurrito]

Re:questions

1. It should fail
2. Compose- changed my mind, I second allow only one

[tordans]

1: read+write
2: allow only one .withValidation and side step the issue.

General: i like the feature.

Curious: Did you compare this with how Tanstack Router ensures value safety?

[franky47]

I have yet to dive into the TanStack router, but from the preview I saw it looks super interesting.

I'd love to provide a similar DX with nuqs, though that would mean tighter integration with Next.js APIs (Link & router), and a way to scan the project for search param definitions to codegen the type-safety.

[franky47]

I'm exploring an alternative API: adding the validation method as an option.

const diceSchema = z.number().integer().min(1).max(6);

const [dice, roll] = useQueryState(
  'dice',
  parseAsInteger.withOptions({
    validate: diceSchema.parse
  })
)

It might not make much difference, but passing it this way allows for a few things:

1. We can use it for validating writes
2. It can also be used on the server cache
3. It solves the open question of compose vs override, by following the same pattern as other options (override).

One issue with this approach is that it makes the Option type generic over the data type, which we could consider a breaking change since it's exported. Though I doubt many people actually import it as-is, and instead infer usage from the hooks API (which does not change). So if going this way I'd be down to mark this as a minor update, with an explicit mention in the release notes.

[sp88011]

With this feature, would it be possible to pass an entire zod object to useQueryStates to initialize the keys, or is this intentionally not the way to go?

Passing an entire zod object would help with reusability because you probably use that same zod elsewhere (e.g. to validate function params). A practical example would be using one zod object to capture every param (pagination, column filters, sort, ...) for a (tanstack) table.

const tableParams = z.object({
page: z.coerce
    .number()
    .transform((v) => v - 1)
    .pipe(z.number().min(0))
    .catch(0),
    
  size: z.coerce
    .number()
    .transform((v) => v.toString())
    .pipe(z.enum(["10", "20", "50", "100"]))
    .transform((v) => Number(v))
    .catch(10),
  
  query: z.string().optional(),
   
   //... etc
   
   })
   
   
   const [tableState, setTableState] = useQueryStates({...parseAsJson.(tableParams.parse}) //something like this?
   

[franky47]

Zod schemas on their own lack the reverse transformation to turn the data type back into a string when writing to the URL, so they can only be used as runtime validators, not serializers/deserializers.

[MartinCura]

In working on this feature, would it benefit to accept any validation library which follows the new Standard Schema spec?

[franky47]

That would be nice, yes. Maybe the parsers should also conform to this standard. (edit: they do now since #965).

The main issue with validation at the moment is that it should also run on updates (not just when parsing), and therefore updates would have to have some sort of error handling mechanism.

[stychu]

is this still being developed ?

[franky47]

It's on the list, but my bandwidth is limited. If/when there are updates, I'll post them in the relevant discussions/issues.

[switz]

It would be really nice if there were a way to type/validate searchParams on a server Page or Layout component. This would remove the need to manually write Promise<{ foo: String }> and then typecast it. The API for it might be awkward, but perhaps just as simple as wrapping a component?

// Describe your search params, and reuse this in useQueryStates / createSerializer:
export const coordinatesSearchParams = {
  latitude: parseAsFloat.withDefault(0),
  longitude: parseAsFloat.withDefault(0),
};

export default wrapParams(coordinatesSearchParams, async function Page({ params, searchParams }) {
  const { latitude } = searchParams; // typed and automatically unwrapped?
});

If a route doesn't match that component it could throw a 404 or 500. Could lead to more rigid routes and less manual typing.

[switz]

Maybe this is better implemented in userland? But would be nice to see some patterns emerge.

[franky47]

Generally, I would recommend using a loader for this:

// Describe your search params, and reuse this in useQueryStates / createSerializer:
export const coordinatesSearchParams = {
  latitude: parseAsFloat.withDefault(0),
  longitude: parseAsFloat.withDefault(0),
};

const loadCoordinates = createLoader(coordinatesSearchParams)

type PageProps = {
  searchParams: Promise<SearchParams> // resolves to { [key: string]: string | string[] | undefined }
}

export default async function Page({ searchParams }: PageProps) {
  const { latitude } = await loadCoordinates(searchParams)
});

But this "automatic (un)wrapping" is something you could achieve with a little userland helper:

import {
  createLoader,
  parseAsFloat,
  type ParserMap,
  type SearchParams,
  type UrlKeys,
  type inferParserType
} from 'nuqs/server'
import type { JSX } from 'react'

const coordinatesSearchParams = {
  latitude: parseAsFloat.withDefault(0),
  longitude: parseAsFloat.withDefault(0)
}

// Currying might make this harder to read, but it makes a clearer separation
// between search params stuff (parser map & options) and the page component to wrap.
export default wrap(coordinatesSearchParams, {
  urlKeys: { latitude: 'lat', longitude: 'lng' }
})(function Page({ searchParams }) {
  const { latitude, longitude } = searchParams
  return (
    <div>
      Latitude: {latitude}, Longitude: {longitude}
    </div>
  )
})

function wrap<Parsers extends ParserMap, PageProps extends {}>(
  parsers: Parsers,
  options?: {
    urlKeys?: UrlKeys<Parsers>
  }
): (
  Page: (
    props: PageProps & { searchParams: inferParserType<Parsers> }
  ) => JSX.Element
) => (
  props: PageProps & { searchParams: Promise<SearchParams> }
) => Promise<JSX.Element> {
  const loader = createLoader(parsers, options)
  return Page =>
    async function WrappedPage(props) {
      const searchParams = await loader(props.searchParams)
      return <Page {...props} searchParams={searchParams} />
    }
}

That being said, unwrapping at this level prevents pre-rendering of the outer page shell. If the search params need to be consumed further down the RSC tree, a better approach is to call the loader but not await its result, and pass the unresolved promise down in nested RSCs surrounded by Suspense boundaries (or use the cache to avoid prop drilling)

It would be really nice if there were a way to type/validate searchParams on a server Page or Layout component

Also note that Layout components do not have access to fresh search params, as they are not rendered dynamically. You'd need a client component in a Layout to access those (either via nuqs or useSearchParams).

[switz]

Good point on the layouts. I’m going to play with this in my apps and see if any interesting patterns arise. If so will share back my findings, maybe some of it can be upstreamed. Thanks for the thoughtful reply and code samples!

[switz]

I went ahead and implemented my own solution to this problem. I created a wrapper called createSearchParams which allows me to generate server side parsing (and validation) and client side hooks from basic schemas. Here's the code:

import { inferParserType, ParserMap, useQueryStates } from 'nuqs';
import { createLoader, createSerializer, LoaderInput, Options } from 'nuqs/server';
import { z } from 'zod/v4';

export function createSearchParams<
  const Schema extends z.ZodTypeAny,
  const Parser extends ParserMap,
>(schema: Schema, parser: Parser) {
  const loadSearchParams = createLoader(parser);

  return {
    schema,
    parser,
    useQueryStates: (options?: Options) => {
      // could add extra stuff to this hook like validation
      const qs = useQueryStates(parser, options);

      const validateSearchParams = (obj: z.infer<Schema>): boolean => {
        try {
          schema.parse(obj);

          return true;
        } catch (err) {
          console.error('Error parsing future validation', err);

          return false;
        }
      };

      return [...qs, validateSearchParams] as const;
    },
    parseAndValidate: async (searchParams: Promise<LoaderInput>) => {
      const parsed = await loadSearchParams(searchParams);
      const result = schema.safeParse(parsed);

      if (result.error) {
        const error = new Error();

        error.cause = z.prettifyError(result.error);

        throw error;
      }

      return parsed;
    },
    buildUrl: (pathname: string, nextSearchParams: Partial<inferParserType<Parser>>) => {
      const serialize = createSerializer(parser);

      const params = serialize(nextSearchParams as any); // not sure the typing here yet

      return [pathname, params].join('');
    },
  };
}

export function combineSearchParams<
  const T extends readonly ReturnType<typeof createSearchParams>[],
>(...params: T) {
  const baseSchema = params.reduce((acc, p) => acc.and(p.schema), z.object({}));
  const baseParser = Object.assign({}, ...params.map((p) => p.parser));

  return createSearchParams(baseSchema, baseParser);
}

To use this, you just setup a zod schema and a nuqs parser in an isomorphic file, e.g. paginationSearchParams.ts:

import { createSearchParams } from '@/lib/createSearchParams';
import { parseAsInteger } from 'nuqs/server';
import { z } from 'zod/v4';

export const paginationSchema = z.object({
  page: z.int().min(1),
  pageSize: z.int().min(1).max(100).nullable(),
});

export const paginationParser = {
  page: parseAsInteger.withDefault(1),
  pageSize: parseAsInteger.withDefault(20),
};

export const paginationSearchParamsLoader = createSearchParams(paginationSchema, paginationParser);

And in your server components you can:

const Explore = async ({ searchParams }: PropsWithSearchParams) => {
  const { page } = await paginationSearchParamsLoader.parseAndValidate(searchParams);
  // page is fully typed and validated, will throw an error if it fails zod validation
};

And in your client components you can:

const Thread = ({ data, id, children }) => {
  const [searchParams, setSearchParams] = paginationSearchParamsLoader.useQueryStates({
    shallow: false,
  });

  // searchParams and setSearchParams are fully typed
});

The big advantage here is you no longer have to pass around your schemas/parsers into each component. You basically create a loader function that works on both the client and the server.

I've migrated most of my app to this new pattern and I find it rather delightful and rigid. In doing so, my code has become a lot simpler to reason about, more logical, and profoundly cleaner. The caveat is that next.js has no way to pass a server-side error to the client-side error.ts or global-error.ts, which is kind of a pain. Once that is possible, the zod validation is all the more valuable as you can then create an error message for the client to understand why the URL didn't pass validation.

At the end of the day, the zod schema is less valuable than I initially thought it would be. Nuqs carries a lot of weight and actually solves most of the problems.

Curious if anyone finds this useful - please feel free to play with it or change it. I'd love to see some new patterns develop.

Thanks to @franky47 for taking a look at this and providing feedback. 👍

[franky47]

If anyone wants to take a stab at typing it properly, I'd be hugely grateful.

One step further would be to use the Standard Schema interface for the validation layer, and have this work with other libs like ArkType and Valibot.

Great work there! 🙌

[reteps]

Loving this RFC! I would personally like an API where I can use a single Zod schema as the source of truth.

Looking at @switz 's workaround, It seems redundant to say:

export const paginationSchema = z.object({
 page: z.int().min(1),
 pageSize: z.int().min(1).max(100).nullable(),
});

export const paginationParser = {
 page: parseAsInteger.withDefault(1),
 pageSize: parseAsInteger.withDefault(20),
};

It would be excellent to be able to write:

export const paginationSchema = z.object({
  page: z.int().min(1).default(1),
  pageSize: z.int().min(1).max(100).default(20).nullable(),
});

and have nuqs understand how to transform this schema into one for serialization, not just deserialization. I'm not familiar with Standard Schema, but a slightly different proposal would be to use a JSON Schema version (See Zod to JSON Schema) in order to convert the types and defaults into a serializer (some variant of createSerializer). I like the original proposal as well with Standard Schema!

[franky47]

Strings and integer based states would indeed be trivial to serialise (assuming the schema doesn't introduce any transform, which Standard Schema does not expose).

The issue has always been on non-trivially serialisable data types (Date being a good example, nuqs has three different parsers for those), even if we were able to introspect the Standard Schema at runtime to detect which SerDe to use (which we can't, see standard-schema/standard-schema#14), we wouldn't have any guarantees of bijectivity between read and write, which would cause data loss.

More on this on the nuqs blog:

https://nuqs.47ng.com/blog/beware-the-url-type-safety-iceberg

[FNDEVVE]

Thank you very much @switz.

Here's a version updated for my needs. It returns the Zod parser result, then checks it against the incoming URL params and redirects in case something is different.

For instance, if I have array of strings like A,B,C, and my Zod schema is like this (it removes wrong items so the parsing doesn't fail):

const somethingEnumSchema = z.enum(somethingValues);

return z.object({
	something: z.preprocess(
		(v) => (Array.isArray(v) ? v.filter((item) => somethingValues.includes(item)) : []),
		z.array(somethingEnumSchema),
	),

and the URL param is something?=A,B,1,2,C the schema would pass, but the URL would still contain invalid data, so I compare the incoming params with the validated ones, redirect in case it's wrong, and return the Zod parser result.

export function createSearchParams<
	const Schema extends z.ZodTypeAny,
	const Parser extends ParserMap,
>(schema: Schema, parser: Parser, options?: LoaderOptions<Parser>) {
	const loadSearchParams = createLoader(parser, options);

	return {
		schema,
		parser,
		useQueryStates: ...,
		parseAndValidateAndRedirect: async (searchParams: Promise<LoaderInput>, pathname: string) => {
			const parsed = await loadSearchParams(searchParams);

			const result = schema.safeParse(parsed);

			if (result.error) {
				const error = new Error();

				error.cause = z.prettifyError(result.error);

				throw error;
			}

			const serialize = createSerializer(parser, options);

			const awaitedParams = await searchParams;

			const incomingParams = new URLSearchParams(
				awaitedParams as Record<string, string>,
			);

			incomingParams.sort()

			const zodSerialized = serialize(result.data as any);

			const zodParams = new URLSearchParams(zodSerialized);

			zodParams.sort();

			if (incomingParams.toString() !== zodParams.toString()) {
				return redirect([pathname, zodSerialized].join(""))
			}

			return result.data;
		},
		buildUrl: ...,
	};
}

That's how the server component page.tsx looks like:

type PageProps = {
	searchParams: Promise<SearchParams>;
};

export default async function Page({ searchParams }: PageProps) {
	const headersList = await headers();

	const pathname = headersList.get("x-pathname");

	const pspl = createSearchParams(schema, parser)

	const data = await pspl.parseAndValidateAndRedirect(searchParams, pathname!);
	
	return ...

}

In order to get pathname in server component I use middleware

import type { NextRequest } from "next/server";
import { NextResponse } from "next/server";

export function middleware(request: NextRequest) {
	const requestHeaders = new Headers(request.headers);
	requestHeaders.set("x-pathname", request.nextUrl.pathname);

	return NextResponse.next({
		request: {
			headers: requestHeaders,
		},
	});
}

Again, amazing work, thank you. Everything is much cleaner now, I am glad noone seen my previous code ;) I like this a lot!

[franky47]

I opened a draft PR of a "unified API" inspired by this thread: #1350

A few main differences:

• The hooks are not included in the object, to allow those definitions to be shared between client & server code (also to let linters detect rules of hooks, they can be touchy when accessing hooks living as properties of an object).
• A composition system with .extend and .pick like what you'd find on Zod schemas
• Standard Schema compatibility (for tRPC, TanStack Router etc)
• .$infer type helper

Feedback is welcome!

[tordans]

Hey! I wanted to share our setup that we use at https://tilda-geo.de/regionen/radinfra?map=14.8/52.4713/13.4426&config=pdqyyt.7h39.16g9vk&v=2 … but never found the time to read the thread and join the conversation.
So for now, I will just post a few bits and links which may or may not be useful for those who follow here :-).

• We handle our migrations in the NextJS middleware
• There are different kinds of migrations happening, not all are related to search params (NUQS); for example we needed a way to migrate "regions" (like projects) https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/middleware.ts#L59
• Part of the migration is, that we check for outdated/unused search params and remove them, so old URLs are not cluttered with unused params https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/middleware.ts#L73-L78
• Before the app renders, we ensure that the map param is valid or use a fallback https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/middleware.ts#L82-L86
• The last step of our middleware cleanup is to sort the search params to always use the same order https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/middleware.ts#L111-L119 – cannot remember from the top of my head why we did this but I consider it good practice for using the URL as sate. There are also SEO considerations (not relevant here) and migrating parts of the URL by replacing strings gets easier as well. The most visible effect is to have a consistent structure while working with the app so I can spot issues quicker…

The core part of our migration is our map config. We store all the active layers and in a config param which we encode to make it smaller. V1 has a JSON-like object here which became too big and was not sharable anymore due to different URL parsing behavior.
https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/middleware.ts#L88-L109

URL state version:
The idea is, that we have a version param that describes the general capabilities and strucutre of the URL. We can use this version to introduce bigger changes.
We migrate to the latest version in https://github.com/FixMyBerlin/tilda-geo/blob/develop/app/src/app/regionen/%5BregionSlug%5D/_hooks/useQueryState/useCategoriesConfig/migrateUrl.ts#L20
There are 2 ATM.

The most magic happens in the config param. Since we want this super short, we do some encoding and store the minimal config on disc. This allows us to migrate from an old to the newest config while preserving as much of the URL state as possible. We take the minimal config and re apply the URL state.
It means that we have to write those configs to disc (and check them in) whenever we change something in the structure of the categories for a given region. This is checked by a test that validates all the configs.

This complex setup is used only for the map state that describes which categories/layers are active. For other states like notes state and "new notes" overlay, we use NUQS out of the box, eg https://tilda-geo.de/regionen/radinfra?map=14.8/52.4713/13.4426&config=pdqyyt.7h39.16g9vk&osmNotes=true&osmNote=14.8/52.4715/13.4411&v=2

Hope this helps someone improving their URL state!

[fakkio]

@franky47

Why not use Zod directly in place of parsers ?

Zod can behave like the parse method: take a string, hydrate it into a JS data type, and run some validation on top while doing so.

However, it does not provide the counterpart serialization function to transform that JS data type back into a string, which is necessary on query state updates.

Also, it would require all Zod schema definitions to be rooted in z.string().transform(s => whatever), which is not what existing application logic would define (schema definitions are likely to be reused or derived from domain-specific definitions).

hi!
I just landed on this library and was reading through this RFC.
Maybe the new Zod codecs could help? https://zod.dev/codecs

[franky47]

I posted this video about connecting Zod codecs to nuqs yesterday:

https://youtu.be/k4lWvklUxUE

Here's the code snippet from the video:

import { createParser } from 'nuqs' // or 'nuqs/server'
import { z } from 'zod' // needs zod@^4.1.0

export function createZodCodecParser<
  Input extends z.ZodCoercedString<string>,
  Output extends z.ZodType
>(
  codec: z.ZodCodec<Input, Output>,
  eq: (a: z.output<Output>, b: z.output<Output>) => boolean = (a, b) => a === b
) {
  return createParser<z.output<Output>>({
    parse(query) {
      return codec.parse(query)
    },
    serialize(value) {
      return codec.encode(value)
    },
    eq
  })
}

[fakkio]

Oh sorry, I missed that — I’ll take a look now. Instantly subscribed!

[theromanticsoul]

+1

[ziyadev]

this works fine btw

const createZodParser = <T extends z.ZodType>(schema: T) =>
  createParser<z.infer<T>>({
    parse: (input) => schema.parse(input),
    serialize: (input) => schema.parse(input),
  });

define your schema like this

const parseAsStringZod = createZodParser(z.string().nonempty());
const parseAsDateZod = createZodParser(z.date());

const searchParams = {
  id: parseAsStringZod,
  expires: parseAsDateZod,
  signature: parseAsStringZod,
};

and use it like this:

export default async function Page({ searchParams }: PageProps) {
  const { id, expires, signature } = await loadSearchParams(searchParams, {
    strict: true, // for strict validation
  });