Skip to content

Handle multiple custom domains for portal routing #6096

Description

@linear-code

Problem

Portal runs as one app on Deploy. Every tenant hits the same deployment, served on portal.unkey.com. We want customers to white-label it, so Fireworks adds developers.fireworks.ai, sets the CNAME we give them, and their end users land on our portal.

Custom domains can't do that today. v2_domains_create_domain authorizes the destination against the caller's own workspace (handler.go:80-95), and a portal tenant doesn't own the destination — not the app, not the environment, not the deployment. Verify then derives the route from FindAppById(dom.AppID), so there's nowhere to put a hostname that has no app of its own.

The rest already works. Contention is workspace_id plus domain, cert issuance is ProcessChallenge{ws, domain}, and neither knows what an app is.

Either way we need our own endpoints (v2_portal_*_domain, authorized against urn.Portal), and either way traffic still goes through frontline_routes unchanged. The question below is where the row lives.

Impact on session URLs

Not this ticket, but this ticket can't ship without it.

createSession builds the portal URL from FindVerifiedCustomDomainByAppID (handler.go:231-245), a LIMIT 1 over a single app_id. That's mostly inert today. Once every portal shares one Unkey-owned app, every portal hostname carries that same app_id and the query starts handing tenants each other's hostnames. So it has to be replaced before portal domains go live, not afterwards.

Direction, for the follow-up ticket: createSession takes an optional hostname, validated as a verified portal domain on that portal in that workspace, falling back to portal.unkey.com when omitted rather than picking one. The hostname then binds the session — stored on portal_sessions, checked at exchangeCode — so a code minted for a branded host can't be burned on a different one.

A — one table

ALTER TABLE custom_domains
  ADD for_workspace_id   varchar(48) NULL,        -- tenant; NULL for app domains
  ADD owner_workspace_id varchar(48)
      AS (COALESCE(for_workspace_id, workspace_id)) STORED NOT NULL,
  ADD UNIQUE (owner_workspace_id, domain);

Portal rows carry our workspace_id / project_id / app_id / environment_id, because that's where the traffic actually goes.

Benefits

  • One claim record, so target_cname and FQDN uniqueness keep working off indexes we already have.
  • UNIQUE (owner_workspace_id, domain) stops a tenant claiming the same hostname as both an app and a portal domain. Caught at insert, not after we've issued a cert.
  • Verify, cert issuance, and the route insert don't change at all.

Trade-offs

  • Isolation turns into a filter we have to remember. Five existing queries need AND for_workspace_id IS NULL, and if we miss one, ListCustomDomainsByEnvironment hands back every tenant's hostname. Nothing enforces it and nothing fails loudly.
  • Three workspace columns on one table. Picking the wrong one is a tenancy bug, not an error.
  • Contention has to move to owner_workspace_id or it silently never fires for portal domains, since both rows carry our workspace_id.
  • We have no generated columns anywhere in pkg/mysql/schema/ today.
flowchart TD
    APP["v2_domains_*_domain"] --> CD
    POR["v2_portal_*_domain"] --> CD
    CD[("custom_domains<br/>app rows and portal rows together")]
    CD --> PIPE["verify · cert · route<br/>all unchanged"]
    PIPE --> FR[("frontline_routes<br/>unique FQDN")]
    CD -.reads.-> DR["deploy queries must filter<br/>AND for_workspace_id IS NULL<br/>every query, forever"]
    CD -.reads.-> PR["portal queries<br/>WHERE portal_id = ?"]
    classDef risk stroke:#e5484d,stroke-width:3px,stroke-dasharray: 5 5
    class DR risk
Loading

One row shape, so the pipeline below never changes. The cost is the dashed node: isolation lives in a predicate every present and future deploy query has to carry, and omitting it fails silently.

B — two tables

CREATE TABLE portal_domains (
  id, workspace_id,          -- the tenant
  portal_id, domain, target_cname, ...
  UNIQUE (workspace_id, domain),
  UNIQUE (target_cname)
);

No project, app, or environment on the row. The portal app's ids come from config when we create the route.

Benefits

  • Isolation is structural. A deploy query can't reach a portal row, so there's no rule to remember.
  • One workspace_id per table and it means one thing.
  • custom_domains doesn't change. No migration, no backfill.
  • Portal-only columns land on the portal table instead of piling up as nullable columns on the shared one.

Trade-offs

  • target_cname has to be unique across both tables. Fix is a separate cnameDomain for portal, so they can't collide by construction.
  • Cross-table contention is a UNION read, so it's advisory. frontline_routes unique FQDN is still the backstop.
  • Verify and cert have to generalize over a source, and the route insert pulls ids from config instead of the row.
  • A tenant claiming one hostname as both an app and a portal domain fails late, on frontline_routes.
flowchart TD
    APP["v2_domains_*_domain"] --> CD[("custom_domains<br/>workspace_id = the customer<br/>target: *.unkey-dns.com")]
    POR["v2_portal_*_domain"] --> PD[("portal_domains<br/>workspace_id = the tenant<br/>target: *.portal.unkey-dns.com")]
    CD --> V["verify, over either source"]
    PD --> V
    V --> C["cert issuance"]
    C --> R["route insert<br/>ids from the row, or from config"]
    R --> FR[("frontline_routes<br/>unique FQDN")]
Loading

The split runs through the endpoints and the row, then both sources share verification, certificates, and routing. No dashed node: isolation is a property of the schema rather than of every query anyone writes next.

I'd go with B. The isolation in A is a rule every future query has to remember, and when someone forgets it, it leaks across tenants and nothing tells us. That's the same shape that already cost us with portal sessions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions