Skip to content

Add change feed API for resources (paged, ordered by change time) #812

Description

@TheTechArch

Background

Consumers of the Resource Registry (e.g. downstream sync/cache jobs) need a way to find out which resources have changed, without re-fetching or diffing the full resource list. We should add a paged "feed" endpoint that lists resources with policy that have changed, ordered by change time, similar in spirit to GET /resourceregistry/api/v1/resource/updated, which already does this for resource↔subject policy relations.

Analysis

The paging pattern to follow

altinn-authentication was checked as the reference for "using the Altinn service utils lib for paging". There is no shared paging NuGet packageAltinn.Authorization.ServiceDefaults (from altinn-authorization-utils) only covers OpenTelemetry/host/Npgsql/Yuniql wiring, not paging. Paging in altinn-authentication is hand-rolled per-repo using three small types: Page<TItem, TToken> / Page<TToken>.Request (continuation-token bookkeeping), Opaque<T> (base64url/JSON-encoded opaque continuation token, IParsable so it binds straight from the query string), and Paginated<T> ({ items, links: { next } } response envelope). Keyset pagination is done by querying pageSize + 1 rows ordered by a monotonic column and using the extra row to build the next-page token.

Good news: this repo already has the same pattern, independently implemented, on ResourceController:

  • src/Altinn.ResourceRegistry/Controllers/ResourceController.cs:531-568GET resource/updated, backed by IResourceRegistryRepository.FindUpdatedResourceSubjects, using Opaque<UpdatedResourceSubjectsContinuationToken> and Paginated<UpdatedResourceSubject>.
  • src/Altinn.ResourceRegistry/Models/Opaque.cs and src/Altinn.ResourceRegistry/Models/Paginated.cs — reusable as-is.
  • The underlying repository query already uses the exact keyset idiom used in altinn-authentication: WHERE (updated_at, resource_urn, subject_urn) > (@since, @resource_urn, @subject_urn) ORDER BY updated_at, resource_urn, subject_urn LIMIT @limit (ResourceRegistryRepository.cs:436-469).
  • ResourceRegistryRepository.Search (ResourceRegistryRepository.cs:44-96) already computes "latest version per resource" via ROW_NUMBER() OVER (PARTITION BY identifier ORDER BY version_id DESC) filtered to rn = 1 — the exact idiom the feed's "one row per resource" requirement below reuses.

So the new endpoint is a matter of applying these already-proven-in-repo patterns to the resources table, not introducing new paging or querying mechanisms.

Resource table as change source

Since migration v0.09-split-resourcetable, resourceregistry.resources is effectively an append-only history table:

  • resource_identifier(identifier PK, created) — one row per logical resource.
  • resources(version_id BIGSERIAL PK, identifier FK, created, modified, serviceresourcejson) — every create/update does an INSERT, never an UPDATE (see ResourceRegistryRepository.CreateResource/UpdateResource), so version_id is a monotonically increasing, collision-free ordering of every change across all resources.
  • current_resources view — DISTINCT ON (identifier) ... ORDER BY identifier, version_id DESC, i.e. latest version per resource.
  • resources.modified (timestamptz) also exists per row, but has no index today and (unlike version_id) can collide/skew across rows, so it's weaker as a cursor.

This means version_id is a ready-made, no-schema-change cursor for a change feed.

Why policy changes needed special handling

The feed as originally sketched was keyed purely on resources.version_id, which is only touched by CreateResource/UpdateResource — i.e. changes to resource metadata (serviceresourcejson). Uploading/changing a resource's XACML policy (POST/PUT /resource/{id}/policy, ResourceController.cs:391-453ResourceRegistryService.StorePolicy, ResourceRegistryService.cs:114-147) never calls CreateResource/UpdateResource and never touches resources. It only (a) writes the policy XML to Azure Blob Storage (PolicyRepository.WritePolicyAsync) and (b) upserts the subjects extracted from that policy into resourceregistry.resourcesubjects (SetResourceSubjects, ResourceRegistryRepository.cs:472-529). So a policy edit is invisible to a version_id-only feed. The design below closes this gap deliberately — see "Design decisions" — rather than leaving it as an unresolved caveat.

Other gaps

  1. Deletes are invisible to the feed. DeleteResource does a hard DELETE FROM resources (ResourceRegistryRepository.cs:179-203) — no row is left behind, so a version_id-based feed will never surface a deletion. resourcesubjects solved the equivalent problem in migration v0.08-resourcesubjects-timestamps-softdelete by adding updated_at + deleted boolean and switching deletes to soft-deletes. We should decide whether resource deletion needs the same treatment, or whether that's out of scope for v1 (see open questions).
  2. ServiceResource.VersionId is int, DB column is bigint/bigserial. If VersionId/version_id becomes the feed cursor, this type needs widening to long to avoid overflow.
  3. No index currently exists on resources.modified; not needed since the design keys off version_id, but called out explicitly so we don't add one by habit.

Design decisions

  1. Only resources with an active policy are listed. A resource that has never had a policy (no rows in resourcesubjects) is filtered out entirely — it's not "active"/delegable, so it's not interesting to feed consumers.

  2. Each resource appears at most once per poll, as its current state — not once per historical version. The feed reports "latest version per resource," reusing the ROW_NUMBER() ... rn = 1 idiom from Search. This also avoids a burst problem: if we'd filtered on "has policy" while still listing full history, a resource with 10 prior metadata-only versions would dump all 10 as a single burst the moment it got its first policy. Filtering + latest-only means it just shows up once, when it currently matters.

  3. All changes are recorded in a dedicated resource_change_log table, and the feed pages on its seq column. (Revised during implementation — the first revision used a "copy the latest version row forward" touch on policy updates, but that duplicated serviceresourcejson per policy write and polluted the resource version history with no-op versions.) The table is resource_change_log(seq BIGSERIAL PK, identifier, changed_at, change_source) with change_source'metadata' | 'policy' | 'deleted', written atomically by every mutation path:

    • CreateResource/UpdateResource insert a 'metadata' entry inside their own transaction/statement (data-modifying CTE), so the log cannot drift from the actual data.
    • StorePolicy calls a new LogPolicyChanged(identifier) repository method after SetResourceSubjects — a plain log insert, no resource-row copying, so a concurrent metadata update can never be affected.
    • DeleteResource logs a 'deleted' entry in the same statement as the delete.

    The migration backfills one entry per existing resource at its latest version (in version order), so pre-existing resources are served from the start. Version history in resources now stays meaning "metadata versions" only, and delete events are recorded — the feed currently excludes resources whose latest change is 'deleted', but exposing deletes to consumers later is just lifting that filter.

Proposal

Path

GET /resourceregistry/api/v1/resource/changes

Named route "changes" (avoids colliding with the existing updated route name used by Url.Link).

Query params

param type notes
token Opaque<long> (optional) continuation cursor, wraps the last-seen version_id
limit int 1–1000, default 1000 — same bounds as updated

No since param, unlike updatedversion_id is a single monotonic integer and doesn't need a timestamp tiebreaker.

Response body

Paginated<ResourceChange> — same envelope shape as the existing updated endpoint:

{
  "links": {
    "next": "https://platform.altinn.no/resourceregistry/api/v1/resource/changes?token=eyJ2ZXJzaW9uSWQiOjEyMzR9&limit=1000"
  },
  "items": [
    { "resourceId": "some-resource-identifier", "changedAt": "2026-07-08T10:15:30Z" },
    { "resourceId": "other-resource-identifier", "changedAt": "2026-07-08T10:16:02Z" }
  ]
}
  • resourceId — the bare identifier (matches how resources/resource_identifier key today; not a full urn:altinn:resource:...).
  • changedAtmodified of that resource's latest version row (bumped by either a metadata edit or a policy "touch," per design decision 3).
  • links.nextnull when there is no further page.
  • Each item represents one resource's current state, not a per-version history entry (per design decision 2) — a resource that changed multiple times between two polls still only appears once.

Paging mechanics

Same mechanic as FindUpdatedResourceSubjects/UpdatedResourceSubjects: fetch limit + 1 rows ordered by version_id; if exactly limit + 1 come back, drop the last one and build the next link from it:

Url.Link("changes", new {
    token = Opaque.Create(new ResourceChangesContinuationToken(last.VersionId)),
    limit
})

giving a URL shaped like .../resource/changes?token=<base64url-opaque>&limit=1000 — the token is opaque to the client, same as today's updated endpoint.

Backing query

Latest version per resource (reusing the Search method's ROW_NUMBER() idiom), filtered to resources with an active policy, keyset-paged on version_id:

WITH latest AS (
    SELECT identifier, version_id, modified,
           ROW_NUMBER() OVER (PARTITION BY identifier ORDER BY version_id DESC) AS rn
    FROM resourceregistry.resources
)
SELECT identifier, version_id, modified
FROM latest
WHERE rn = 1
  AND version_id > @cursor
  AND EXISTS (
      SELECT 1 FROM resourceregistry.resourcesubjects rs
      WHERE rs.resource_urn = 'urn:altinn:resource:' || latest.identifier
        AND rs.deleted = false
  )
ORDER BY version_id
LIMIT @limit + 1

(A view-based current_resources alternative was considered, but pushing version_id > @cursor through a DISTINCT ON view doesn't reliably let Postgres skip old versions — the explicit CTE, matching the Search method's existing pattern, is the safer bet for a paged query.)

Open questions

  • Delete events are now recorded in resource_change_log, and deleted resources drop out of the feed. Still open: should the feed expose delete events to consumers (e.g. a changeType/deleted field), or is silent exclusion enough for v1? Exposing them later is just lifting the change_source <> 'deleted' filter.
  • Silent disappearance: if a resource's last active policy subject is soft-deleted, it drops out of the EXISTS filter going forward. Consumers can't distinguish "never had policy" from "policy was removed" purely from feed absence. Acceptable for v1?
  • Auth: should this endpoint be open like resourcelist/Search, or scoped like updated?

Status

Implemented in PR #817:

  • Migration v0.10-resource-change-log: resource_change_log table + backfill of existing resources
  • 'metadata' log entries written atomically inside CreateResource/UpdateResource
  • 'deleted' log entries written atomically inside DeleteResource
  • LogPolicyChanged called from ResourceRegistryService.StorePolicy after SetResourceSubjects
  • Keyset-paged feed query (latest change per resource + has-policy + not-deleted) on seq
  • GET resource/changes endpoint using Opaque<long>/Paginated<ResourceChange>
  • Tests: paging, ordering, policy update bumps feed position, resources without policy excluded, deleted resource excluded, invalid limit

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions