URI Parsing Library for Pony

[SeanTAllen]

this originated on ponylang/lori_http_server and was transfered here to ponylang/uri

Context

lori_http_server passes raw request-target strings to handlers as String val. Discussion ponylang/lori_http_server#2 explicitly deferred URI parsing: "Full RFC 3986 parsing can be a separate package." This plan covers building that capability as a uri subpackage within http_server/.

ponylang/http_server includes URL parsing (URL class, URLEncode primitive) but has design issues:

• Error handling uses partial functions (?) rather than error-as-data types
• All URL fields are var — mutable during construction via ref methods
• No query parameter parsing (query is a raw string)
• No path segment decomposition
• join() is declared but unimplemented
• Coupled to the http_server package
• check_scheme() fails to validate that the first character is alpha (allows 1foo: as a scheme)

Package Location

The uri package lives at http_server/uri/ as a subpackage of http_server, following the same pattern as semver's subpackages (semver/range, semver/solver, etc.). If the package proves useful beyond this project, it can be extracted into its own repo later.

corral.json's packages array will include "http_server/uri" to declare it as a provided package. In these initial phases, http_server does not import uri — the Handler trait continues to pass raw String val. Integration into http_server's interface can follow once the uri package is built.

Scope

In scope

• Parse RFC 3986 URI-references into structured components
• Percent-encoding and percent-decoding
• Query parameter parsing (application/x-www-form-urlencoded key-value pairs)
• Path segment decomposition
• URI string reconstruction from components

Out of scope (can be added later)

• URI resolution (RFC 3986 §5 — combining base + relative reference)
• IRI support (RFC 3987)
• Normalization beyond structural decomposition
• URI template expansion (RFC 6570)

Design

Error Types

Each operation defines its own error vocabulary. Structural parsing and percent-encoding have separate error types since they come from different operations.

// Structural parsing errors (returned by ParseURI / ParseURIAuthority)
primitive InvalidPort is Stringable
  """Port is non-numeric or exceeds U16 range."""
  fun string(): String iso^ => "InvalidPort".clone()

primitive InvalidHost is Stringable
  """
  Malformed IP-literal (unmatched brackets, illegal characters in IPv6
  address, or invalid IPvFuture syntax).
  """
  fun string(): String iso^ => "InvalidHost".clone()

type URIParseError is (InvalidPort | InvalidHost)

// Percent-encoding error (returned by PercentDecode, ParseQueryParameters,
// PathSegments)
primitive InvalidPercentEncoding is Stringable
  """Percent-encoded sequence is truncated or contains non-hex digits."""
  fun string(): String iso^ => "InvalidPercentEncoding".clone()

Core Types

All parsed types are class val — immutable after construction, fields are let. Construction is through factory primitives that return (T | Error), not partial constructors.

URI — the top-level parsed URI-reference:

class val URI is (Stringable & Equatable[URI])
  """
  A parsed RFC 3986 URI-reference.

  Components are stored in their percent-encoded form as they appeared in
  the input. Delimiter characters (`:` after scheme, `//` before authority,
  `?` before query, `#` before fragment) are stripped — use `string()` to
  reconstruct the full URI with delimiters.

  `string()` reconstruction follows RFC 3986 section 5.3. Components that
  are `None` are omitted entirely (no delimiter). A component that is an
  empty `String` (present but empty) includes its delimiter — e.g.,
  `query` of `""` produces a trailing `?` with no value. This distinction
  allows faithful reconstruction of inputs like `http://example.com/path?`
  (empty query present, `query = ""`) vs. `http://example.com/path`
  (no query at all, `query = None`).

  Equality is structural: two URIs are equal when all their stored
  (percent-encoded) components are equal. No normalization is applied
  before comparison.
  """
  let scheme: (String | None)           // None for relative references
  let authority: (URIAuthority | None)  // None when no authority present
  let path: String                      // always present (may be empty string)
  let query: (String | None)            // None when no ? in input
  let fragment: (String | None)         // None when no # in input

URIAuthority — the parsed authority component:

class val URIAuthority is (Stringable & Equatable[URIAuthority])
  """
  A parsed URI authority component (RFC 3986 section 3.2).

  Format: [userinfo@]host[:port]

  The host may be empty — this occurs in URIs like `file:///path` where
  the authority is present (indicated by `//`) but contains no host.
  IP-literals (IPv6 and IPvFuture) include their brackets in the host
  string (e.g., `[::1]`).
  """
  let userinfo: (String | None)  // None when no @ present
  let host: String               // may be empty; IPv6/IPvFuture includes brackets
  let port: (U16 | None)         // None when no port specified

Factory Primitives

ParseURI — parse a URI-reference:

primitive ParseURI
  fun apply(raw: String val): (URI val | URIParseError val) =>
    """
    Parse a URI-reference (RFC 3986 section 4.1) into structured components.

    Handles both absolute URIs (`http://host/path?query`) and relative
    references (`/path?query`). Components are stored percent-encoded.

    The parsing algorithm follows RFC 3986 section 3: scan for `:` before
    any of `/?#` to identify the scheme, then check for `//` to identify
    authority, then extract path (up to `?` or `#`), query (between `?`
    and `#`), and fragment (after `#`). Scanning operates on literal
    delimiter characters only — percent-encoded delimiters (`%2F`, `%3F`,
    `%23`) are not treated as structural boundaries.

    Scheme validation: the scheme must start with a letter and contain
    only letters, digits, `+`, `-`, and `.` (RFC 3986 section 3.1).

    The asterisk-form (`*`) used by HTTP OPTIONS is a valid relative
    reference under RFC 3986 — it parses as a URI with path `*` and no
    other components.
    """

ParseURIAuthority — parse an authority string directly:

primitive ParseURIAuthority
  fun apply(raw: String val): (URIAuthority val | URIParseError val) =>
    """
    Parse a standalone authority string ([userinfo@]host[:port]).

    Use this for HTTP CONNECT request-targets (authority-form) where the
    request-target is `host:port` without a scheme or `//` prefix.
    `ParseURI` would misinterpret `host:port` as `scheme:path`.
    """

Percent-Encoding Utilities

primitive PercentDecode
  fun apply(input: String val): (String val | InvalidPercentEncoding val) =>
    """
    Decode all percent-encoded sequences in the input.

    Does NOT decode `+` as space — that convention belongs to
    application/x-www-form-urlencoded (handled by ParseQueryParameters),
    not RFC 3986 percent-encoding.
    """

primitive PercentEncode
  fun apply(input: String val, part: URIPart): String val =>
    """
    Percent-encode characters that are not allowed unencoded in the
    specified URI component per RFC 3986.

    For `URIPartHost`, this applies to reg-name hosts only. IP-literals
    (IPv6 addresses and IPvFuture in brackets) have their own syntax and
    should not be percent-encoded — pass them through unchanged.
    """

Component type tags for encoding rules:

primitive URIPartUserinfo
primitive URIPartHost
primitive URIPartPath
primitive URIPartQuery
primitive URIPartFragment

type URIPart is
  ( URIPartUserinfo | URIPartHost | URIPartPath
  | URIPartQuery | URIPartFragment )

Query Parameter Parsing

Separate from the core URI type — RFC 3986 doesn't define query parameter structure. This implements the application/x-www-form-urlencoded format used by HTML forms and virtually all HTTP query strings.

primitive ParseQueryParameters
  fun apply(query: String val)
    : (Array[(String val, String val)] val | InvalidPercentEncoding val)
  =>
    """
    Parse a query string into key-value pairs. Splits on `&`, then on
    first `=`. Decodes `+` as space and percent-decodes both keys and
    values.

    Duplicate keys are preserved in order. Keys without `=` get an
    empty-string value.

    The `;` separator (from an older HTML convention) is not supported —
    it was dropped from the WHATWG URL Standard and is rarely encountered
    in practice.
    """

Path Segment Decomposition

primitive PathSegments
  fun apply(path: String val)
    : (Array[String val] val | InvalidPercentEncoding val)
  =>
    """
    Split a URI path on `/` delimiters, then percent-decode each segment.

    A leading `/` produces an empty first segment (distinguishes absolute
    from relative paths). Splitting happens before decoding, so a
    percent-encoded `/` (%2F) within a segment is preserved as a literal
    slash in the decoded segment, not treated as a delimiter.
    """

Usage from Application Code

In these phases, http_server does not yet import uri — the Handler trait continues to pass uri: String val. Application code can import the uri subpackage directly to parse the raw string. Typical usage in a handler:

use http_server = "http_server"
use uri = "http_server/uri"

// In a Handler's request() method:
fun ref request(method: http_server.Method, uri_str: String val,
                version: http_server.Version,
                headers: http_server.Headers val) =>
  match uri.ParseURI(uri_str)
  | let u: uri.URI val =>
    // u.path for routing, u.query for parameters
    match u.query
    | let q: String val =>
      match uri.ParseQueryParameters(q)
      | let params: Array[(String val, String val)] val =>
        // use params
      end
    end
  | let err: uri.URIParseError val =>
    // unlikely for valid HTTP request-targets, but handle gracefully
  end

For CONNECT requests (authority-form host:port):

match uri.ParseURIAuthority(uri_str)
| let auth: uri.URIAuthority val =>
  // auth.host, auth.port
end

Implementation Phases

Phase 1: Percent-Encoding

Everything else depends on this.

Files:

• http_server/uri/percent_encoding.pony — PercentDecode, PercentEncode, URIPart types, InvalidPercentEncoding
• http_server/_test.pony (modified) — add use uri = "./uri" and uri.Main.make().tests(test) delegation call

Build integration: The http_server/uri/_test.pony Main actor follows the stdlib delegation pattern (new create(env) for standalone use, new make() => None + fun tag tests() for delegation). http_server/_test.pony calls uri.Main.make().tests(test) to include URI tests in the existing test binary.

Testing (http_server/uri/_test_percent_encoding.pony):

• Property: PercentDecode(PercentEncode(s, part)) roundtrips for arbitrary strings and each URI part
• Property: PercentEncode output for each part contains only RFC 3986-legal characters for that part
• Property: invalid percent sequences (truncated %X, non-hex %GG, trailing %) produce InvalidPercentEncoding
• Property: mixed valid/invalid generator — PercentDecode succeeds iff input is the valid variant
• Example: known encodings from RFC 3986 section 2.1
• Counterfactual checks on all new assertions before reporting success

Run: make ssl=3.0.x

Phase 2: URI Parsing

Structural decomposition of URI-references.

Files:

• http_server/uri/uri.pony — package docstring and URI class with string() and eq() methods
• http_server/uri/uri_authority.pony — URIAuthority class with string() and eq() methods
• http_server/uri/parse_uri.pony — ParseURI factory
• http_server/uri/parse_uri_authority.pony — ParseURIAuthority factory
• http_server/uri/uri_parse_error.pony — InvalidPort, InvalidHost, URIParseError union
• http_server/uri/_mort.pony — _Unreachable, _IllegalState (same issue URL as http_server — both are in this repo). This is an intentional duplication of http_server/_mort.pony because these primitives are package-private (_-prefixed) and cannot be shared across packages in Pony

The package docstring in http_server/uri/uri.pony should guide users toward the right entry point: ParseURI for standard URI-references, ParseURIAuthority for HTTP CONNECT authority-form targets, PercentDecode/PercentEncode for encoding operations, and ParseQueryParameters/PathSegments for higher-level query and path access.

Equality implementation note: URIAuthority.eq() must be implemented first since URI.eq() depends on it. Both must handle (String | None) field comparisons (match on both sides, equal when both None or both String with equal content). URI.eq() also matches on (URIAuthority | None) for the authority field, delegating to URIAuthority.eq() when both are present.

Generator strategy for URI property tests:
Build generators compositionally from the RFC 3986 grammar:

1. Scheme generator: letter + sequence of (letter | digit | + | - | .)
2. Host generator: oneof across reg-name (unreserved/pct-encoded chars), IPv4 dotted-quad, IPv6 literal in brackets
3. Port generator: U16 range
4. Authority generator: compose optional userinfo, host, optional port
5. Path generator: sequence of / + pchar segments
6. Query generator: pchar + (/ | ?) characters
7. Fragment generator: pchar + (/ | ?) characters
8. URI generator: compose optional scheme, optional authority, path, optional query, optional fragment with correct delimiters

Invalid generators negate specific rules: port > 65535, unmatched IPv6 brackets, non-hex in port, etc.

Testing (http_server/uri/_test_parse_uri.pony, http_server/uri/_test_parse_uri_authority.pony):

• Property: roundtrip — for generated valid URIs, ParseURI(uri.string()) produces an equal URI (using Equatable)
• Property: generated absolute URIs with all components parse correctly, each field matches the generated input
• Property: generated relative references (/path, /path?query, /path?query#frag) parse with scheme = None and authority = None
• Property: valid/invalid/mixed triad for URI parsing — valid URIs always parse successfully, invalid URIs (bad port, bad host) always produce the correct error type, mixed generator succeeds iff input is the valid variant
• Property: valid/invalid/mixed triad for authority parsing
• Property: generated invalid ports (non-numeric, > 65535) produce InvalidPort
• Property: generated invalid schemes (starting with digit, containing illegal chars) parse with scheme is None — they are treated as relative references, not as errors
• Property: generated malformed IPv6 hosts (unmatched brackets, illegal chars) produce InvalidHost
• Example: RFC 3986 section 1.1.2 examples (ftp://ftp.is.co.za/..., ldap://[2001:db8::7]/..., etc.)
• Example: HTTP request-target forms — origin-form /index.html?page=1, absolute-form http://www.example.org/pub/WWW/TheProject.html, asterisk-form *
• Example: edge cases — empty string (valid empty relative reference), query without value (?key), empty query present (/path?), fragment only (#frag), authority without port, empty authority (file:///etc/hosts)
• Example: percent-encoded delimiters not treated as structural — http://example.com/a%2Fb?c%3Fd should parse as path /a%2Fb, query c%3Fd
• Counterfactual checks on all new assertions before reporting success

Run: make ssl=3.0.x

Phase 3: Path and Query Utilities

Higher-level operations built on the core types.

Files:

• http_server/uri/path_segments.pony — PathSegments primitive
• http_server/uri/query_parameters.pony — ParseQueryParameters primitive

Testing (http_server/uri/_test_query_parameters.pony, http_server/uri/_test_path_segments.pony):

• Property: PathSegments count equals the number of /-delimited parts in the generated path
• Property: percent-encoding and joining segments with / reconstructs the original path
• Property: ParseQueryParameters roundtrips — generate key-value pairs, serialize as k=v&k2=v2, parse back, verify match
• Property: + in query values decodes as space
• Property: keys without =, empty values, empty keys all handled
• Property: valid/invalid/mixed triad for ParseQueryParameters — valid query strings succeed, strings with bad percent-encoding fail
• Example: a=1&b=2 produces [("a","1"), ("b","2")]
• Example: key=hello+world produces [("key", "hello world")]
• Example: a=1&a=2 produces [("a","1"), ("a","2")] (duplicate keys preserved)
• Example: empty string produces empty array
• Counterfactual checks on all new assertions before reporting success

Run: make ssl=3.0.x

Phase 4: Integration with http_server

Wire the URI package into the example and documentation. http_server does not yet import uri in these phases — that integration can follow separately.

• Add "http_server/uri" to corral.json's packages array to declare it as a provided package
• Update examples/basic/main.pony to demonstrate URI parsing and query parameter access (using use uri = "../../http_server/uri")
• Update Handler docstrings to show how to use ParseURI with the raw uri string
• Update CLAUDE.md file layout section with the new http_server/uri/ subpackage and its files

File Layout

http_server/uri/                      # new subpackage inside http_server/
  _mort.pony                          # _Unreachable, _IllegalState
  uri.pony                            # package docstring + URI class
  uri_authority.pony                  # URIAuthority class
  percent_encoding.pony               # PercentDecode, PercentEncode, URIPart types,
                                      # InvalidPercentEncoding
  parse_uri.pony                      # ParseURI factory
  parse_uri_authority.pony            # ParseURIAuthority factory
  uri_parse_error.pony                # error types and URIParseError union
  query_parameters.pony               # ParseQueryParameters
  path_segments.pony                  # PathSegments
  _test.pony                          # \nodoc\ Main actor (stdlib delegation
                                      # pattern: create + make + tests)
  _test_percent_encoding.pony         # percent-encoding tests
  _test_parse_uri.pony                # URI parsing tests
  _test_parse_uri_authority.pony      # authority parsing tests
  _test_query_parameters.pony         # query parameter tests
  _test_path_segments.pony            # path segment tests

[SeanTAllen]

This is complete including the out-of-scope items.