Skip to content

Fetchers 2.0 Re-design #4860

Description

@karwaankit32

Config Fetcher — Refactor (Phase 1)

1. Summary

Refactor the Prebid Server account configuration fetch path to eliminate per-request JSON parsing, default-merging, and map-building, and to collapse concurrent cache misses into a single upstream call per pod. The new design caches an immutable, fully-derived *config.Account with negative caching and per-backend metrics.

Phase 1 is scoped to account config only implementation but provides a path for GVL and stored req/response re-structuring.

2. Background

Current State

flowchart LR
    subgraph Stored[stored_requests]
        S1["Fetch<br>db, http, file"] --> S2["Cache<br>raw JSON, TTL"]
        S3["Refresh: poll for changes <br> + invalidate"] --> S2
    end
    subgraph GVL[GDPR vendor list]
        G1["Fetch http"] --> G2["Cache<br>ALL versions preloaded"]
        G3["Refresh: on-demand <br> rate-limited ~10 min"] --> G2
    end
    subgraph CUR[currency]
        C1["Fetch http"] --> C2["Cache<br>rates in atomic value"]
        C3["Refresh: periodic timer"] --> C2
    end
    subgraph ACC[account hot path]
        A1["Fetch raw JSON"] --> A2["Parse + merge defaults +<br> derive EVERY REQUEST <br> no typed cache"]
    end
Loading

Problems: 3 different refresh loops, GVL preloads everything (startup + ~2/3 memory), account does unmarshal+merge+derive on every request.

Inefficiency — Account Fetcher

Every OpenRTB auction served by Prebid Server resolves an account configuration via account.GetAccount (see account/account.go), which calls a chain of:

account.GetAccount
  └── stored_requests.AccountFetcher.FetchAccount   (interface)
        └── fetcherWithCache.FetchAccount           (byte-cache wrapper)
              └── HttpFetcher.FetchAccount          (upstream config service)
                    └── jsonpatch.MergePatch(defaults, accountJSON)

The cache stores raw merged JSON bytes. On every request — cache hit or miss — GetAccount then runs:

  • jsonutil.UnmarshalValid(accountJSON, &config.Account) — full JSON decode.
  • setDerivedConfig(account) — allocates PurposeConfigs, VendorExceptionMap, BasicEnforcementVendorsMap, etc.

Account configs are growing in size as PSP adopts modularity for Bidder Optimization, so this per-request cost scales linearly with config size × QPS.

Inefficiency — GVL Fetcher

At startup, every Prebid Server instance preloads its GDPR vendor-list cache by fetching every historical version of the Global Vendor List — for each supported spec version (2 and 3), it loops from the first list version up to the latest and downloads them all:

NewVendorListFetcher (startup)
  └── for spec in {2, 3}:
        └── for list = first .. latest:      ← every version ever published
              └── HTTP GET vendor-list-v{list}.json
              └── ParseEagerly → api.VendorList
              └── cacheSave   (never expires, never evicted)

This has two costs:

  • Startup fan-out — hundreds of blocking, sequential HTTP round-trips (no parallelism) before the fetcher is ready, each with a full parse.
  • Unbounded memory — the cache never expires or evicts, so all versions stay resident for the life of the process, even though live traffic only references the latest few.

Both grow linearly with the total number of published GVL versions, while the actually-used working set stays roughly constant (the most recent handful). The redesign bounds preload to the latest N versions per spec, serving rare older versions lazily on demand.

3. Proposed model — Source × Cache × Transform (+ optional Refresh)

flowchart LR
    ID["id / ids"] --> CACHE{"Cache<br/>Get"}
    CACHE -- hit --> OUT["typed value V"]
    CACHE -- miss --> SRC["Source<br/>pull raw bytes"]
    SRC --> TR["Transform<br/>unmarshal + <br>overlay defaults + derive"]
    TR --> SAVE["Cache.Save V"]
    SAVE --> OUT
    REFRESH["Refresh<br/>none or preload"] -. "seeds/expires" .-> CACHE
Loading

Key rule: the cache stores the composed typed value V, not raw JSON. Transform runs once per id at insert time. Hot path on a hit = pure map lookup, no unmarshal, no json merge.

Source, Transform and Cache above are generic interfaces — the engine is generic over [K, V] and never names a concrete backend or value type:

// Source: I/O only. Returns undecoded bytes. Knows nothing about V.
// BATCH-keyed — a single lookup is a one-element slice. (http_fetcher.FetchAccounts /
// FetchRequests / FetchResponses are already batch; the single-key methods are the shims.)
// GVL implements this too, but loops internally (one URL per version — no batch collapse).
type KeyedSource[K comparable] interface {
    Fetch(ctx context.Context, keys []K) (map[K]json.RawMessage, []error)
}

// KeylessSource: same http transport as KeyedSource, just a fixed URL and no key (one document).
// NOT a separate backend — a thin shape adapter over the same GET+read-body, used by CURRENCY only.
type KeylessSource interface {
    Fetch(ctx context.Context) (json.RawMessage, error)
}

// Transformer: raw bytes -> typed V. Runs ONCE per value at cache insert. Uniform for all four.
// (account = unmarshal+defaults+derive; GVL = ParseEagerly; currency = Rates unmarshal;
//  stored = Identity(), a typed no-op that returns the bytes unchanged.)
type Transformer[V any] interface {
    Transform(raw json.RawMessage) (V, error)
}

// Cache: keyed store of composed typed V. Authoritative def (Get/GetBatch/Save/SaveNegative/
// Invalidate + the `state` enum). Used by account, stored, GVL.
type Cache[K comparable, V any] interface

// Snapshot: keyless single-value box. Used by CURRENCY only. Read = pure atomic load.
type Snapshot[V any] interface {
    Get() V        // current value, or fallback if never filled
    Store(val V)
}

// Refresher: WHEN the container is populated / kept fresh. Two families, matched to the two
// containers. The only slot with a lifecycle.
type KeyedRefresher[K comparable, V any] interface {   // none | preload | delta-poll
    Start(ctx context.Context, s KeyedSource[K], t Transformer[V], c Cache[K, V])
    Stop()
}
type SnapshotRefresher[V any] interface {              // periodic (currency)
    Start(ctx context.Context, s KeylessSource, t Transformer[V], snap Snapshot[V])
    Stop()
}

Concrete backends (HTTPSource, SQLSource, FileSource, EmptyFetcher) each return a KeyedSource; currency returns a KeylessSource. The value type is chosen by the Transformer (AccountTransformer*config.Account, GVLTransformerapi.VendorList, CurrencyTransformer*Rates, Identityjson.RawMessage). Two constructors wire them:

  • New[K, V](KeyedSource[K], Transformer[V], Cache[K, V], KeyedRefresher[K, V]) → account, stored, GVL.
  • NewSnapshot[V](KeylessSource, Transformer[V], Snapshot[V], SnapshotRefresher[V]) → currency.

The split is not cosmetic: New puts a loader on the read path (miss → fetch → transform → save with negatives); NewSnapshot is a read-only view over a box the periodic refresher fills — reads never fetch, so no negatives, no key.

The three closed axes

A design is one pick from each axis.

  • Source (where the bytes come from)
    • http — fetch raw JSON from a remote HTTP endpoint
    • sql — read a JSON blob column from a database
    • file — read JSON from a local file
    • empty — null-object; no backend, every lookup returns not-found
  • Cache (retention policy)
    • nil — no caching, every Get hits the Source
    • ttl — entries expire after a configured interval
    • load-once — populated once (e.g. at startup), never expires
    • negative decorator — optionally layered on any of the above to cache definitive not-found verdicts
    • snapshot — keyless single-value box, overwritten wholesale on refresh
  • Refresh (freshness)
    • ttl — freshness via TTL expiry only
    • none — never refresh
    • preload — warm the cache up front
    • delta-poll — future
    • periodic — timer re-fetches the whole value and overwrites (currency; also GVL's rate-limited refresh)

Read-path — Account (TTL-only)

sequenceDiagram
    participant Auc as Auction (GetAccount)
    participant K as cachekit(string,*Account)
    participant C as TTL+negative cache
    participant S as Source (http/sql)
    participant T as Transform (parseAndDerive)

    Auc->>K: Get(accountID)
    K->>C: lookup
    alt positive hit (not expired)
        C-->>K: *Account
    else negative hit (unknown id, not expired)
        C-->>K: NotFound (no backend call)
    else miss / expired
        K->>S: fetch raw JSON
        alt found
            S-->>T: raw
            T-->>K: *Account (defaults overlaid + derived)
            K->>C: Save(id, *Account, ttl)
        else 404 / 400 (definitive)
            S-->>K: NotFound
            K->>C: Save negative (negTTL)
        else 5xx / timeout / network
            S-->>K: error (DO NOT cache)
        end
    end
    K-->>Auc: *Account or NotFound
Loading

Notes:

  • Freshness = TTL expiry only. TTL value sits in the 3h min / 6–12h target band.
  • Negative cache policy from POC: cache only 404/NotFound + 400; never 401/403/429/5xx/timeout/decode.
  • Direct-DB companies = swap Cache to nil → every Get hits Source live (no other change).

Read-path — GVL (preload latest N, fixed N)

sequenceDiagram
    participant Init as Startup
    participant K as cachekit(gvlKey, VendorList)
    participant C as load-once cache
    participant S as gvl http Source

    Init->>S: fetch latest version number
    loop latest-N .. latest
        Init->>S: fetch version
        S-->>C: Save(specVer,listVer to VendorList) [never expires]
    end
    Note over C: only N recent versions resident (not all)

    par at request time
        K->>C: Get(specVer,listVer)
        alt in cache
            C-->>K: VendorList
        else older than N (rare)
            K->>S: on-demand fetch (future: rate-limited)
            S-->>C: Save
            C-->>K: VendorList
        end
    end
Loading

Note: Startup fetches N versions instead of all; memory drops from "all versions" to N.

Startup wiring (composition, builder-style)

flowchart TB
    CFG["config: source <br> type, cache type, <br> ttl, N, refresh"] --> BUILD["cachekit.New"]
    PICKS["pick Source impl<br/>pick Cache impl<br/>attach Transform<br/>pick Refresh"]
    BUILD --> PICKS
    PICKS --> FET["returns Source-like fetcher<br/>caller sees only Get"]
    FET --> ACC["account.GetAccount"]
    FET --> GVL["gdpr permissions"]
    FET --> STORED["stored_requests"]
Loading

Illustrative construction (pseudo-code):

accountFetcher := cachekit.New[string, *config.Account](
    cachekit.HTTPSource(client, accountsURL, auth),
    AccountTransformer{defaults: cfg.AccountDefaultsJSON()},
    cachekit.LRU(maxEntries, ttl).WithNegative(negMax, negTTL),
    cachekit.NoRefresh(),
)
// read: acct, err := accountFetcher.Get(ctx, "pub-42")

storedResponseFetcher := cachekit.New[string, json.RawMessage](
    cachekit.SQLSource(db, respQueryTemplate),
    cachekit.Identity(),
    cachekit.LRU(maxEntries, ttl),
    cachekit.DeltaPoll(interval, changeSource),
)
// read: m, errs := storedResponseFetcher.GetBatch(ctx, ids)   // batch
// (stored request+imp share one source that fills two caches; same shape)

gvlFetcher := cachekit.New[gvlKey, api.VendorList](
    cachekit.HTTPSource(client, gvlURLMaker),                  // one URL per version (loops)
    GVLTransformer{},
    cachekit.LoadOnce(),
    cachekit.Preload(n),
)
// read: list, err := gvlFetcher.Get(ctx, gvlKey{spec, ver})

// Currency — keyless: snapshot box, periodic timer.
currencyFetcher := cachekit.NewSnapshot[*currency.Rates](
    cachekit.HTTPDocSource(client, syncSourceURL),            // same http source, fixed URL, no key
    CurrencyTransformer{},
    cachekit.Box(currency.NewConstantRates(), staleWindow),   // atomic value + constant-rates fallback
    cachekit.Periodic(refreshInterval),
)
// read: rates := currencyFetcher.Get()
//       rate, err := rates.GetRate("USD", "EUR")             // unchanged on top

Type mapping

Subsystem Key Source Transform (compose) Cache Refresh Read
Account string http/sql unmarshal+defaults+derive ttl+negative none Get
GVL (spec,list) gvl-http ParseEagerly → VendorList load-once preload Get
Stored req/imp/resp string sql/http Identity (raw JSON) ttl delta-poll GetBatch
Currency (keyless) http (no key) Rates unmarshal snapshot (box) periodic Get()

4. Low-Level Design

Cache type is selectable per fetcher via config; default is lru. There is exactly one Cache[K,V] interface and the Fetcher / callers are identical regardless of policy.

accounts:
  v2_enabled: true # default false
  cache:
    type: lru            # none | lru | unbounded   (default: lru)
    max_entries: 50000   # required when type=lru
    ttl_seconds: 3600
    negative:
      enabled: false     # OPT-IN, default OFF
      max_entries: 10000 # separate budget from positives
      ttl_seconds: 60
  • Note — v2_enabled is the feature flag that will need to be enabled to switch traffic/workflow to fetchers 2.0. By default, it will be set to false.
type Behavior Use case
none always fetch from source, retain nothing direct-DB tenants needing per-request fresh
lru bounded read-through (DEFAULT) account, GVL, stored-data via http/db
unbounded authoritative mirror, no eviction stored-data poll-mirror only

Data structures

Every cached value is the typed deserialization of a JSON source; V is chosen per subsystem:

subsystem JSON source cached V Transform
Account account JSON *config.Account unmarshal + merge defaults + derive (once)
GVL vendor-list JSON api.VendorList ParseEagerly (once)
Stored req/imp/resp stored JSON json.RawMessage identity

Note: Stored request stays bytes on purpose — it's MergePatched against the live request per auction, so there is no stable typed value to cache (same split as prebid-server-java: typed Account, raw-JSON stored data).

type state uint8
const (
    stEmpty    state = iota // in neither store (never seen / evicted / expired)
    stPositive              // found in the positive store
    stNegative              // found in the negative store (negatives enabled only)
)

// positive store value: the composed TYPED value + its expiry
type entry[V any] struct {
    v       V         // composed TYPED value
    expires time.Time // zero == never expires (load-once / mirror mode)
}

type Cache[K comparable, V any] interface {
    Get(key K) (v V, st state)          // st derived from which store answered (account, GVL)
    GetBatch(keys []K) (found map[K]V, missing []K) // stored req/resp — partition hits vs misses
    Save(key K, v V, ttl time.Duration) // ttl==0 ⇒ never expires  → positive store
    SaveNegative(key K, ttl time.Duration) //                       → negative store
    Invalidate(key K)
}

// Keyless container for currency (§8.4). Not a Cache — no keys, no negatives, no miss path.
type Snapshot[V any] interface {
    Get() V        // current value, or constant-rates fallback if never filled
    Store(val V)   // written only by the periodic refresher
}

Read: Get checks the positive store first (→ stPositive), then the negative store (→ stNegative), else stEmpty.

Worked example (account cache, negatives ON)

Four sequential Get calls; the two stores shown after each. state is recomputed each call from which store answered — it is never persisted.

# call actual value cached? negative value cached? state action taken
1 Get("pub-123") (real) no no stEmpty fetch → 200 → Save("pub-123", acct, 1h)
2 Get("pub-123") again yes stPositive return acct — no backend, no unmarshal
3 Get("pub-999") (absent) no no stEmpty fetch → 404 → SaveNegative("pub-999", 60s)
4 Get("pub-999") again no yes stNegative return NotFound — no backend call

Three impls: live (no-op), lru (default — thin wrapper over hashicorp/golang-lru/v2 expirable.LRU), unbounded (map + RWMutex, poll-mirror only). Negatives are a separate bounded store (own small cap + short negTTL), so an unknown-key flood can never evict real data; instantiated only when negative.enabled: true. The reason for using hashicorp/golang-lru/v2 vs existing coocood/freecache is that the latter doesn't support typed values to be cached.

The Fetcher

type Fetcher[K comparable, V any] struct {
    source    KeyedSource[K]      // BATCH raw fetch (http/db/file/empty); single = one-element slice
    transform Transformer[V]      // bytes → typed V, ONCE at insert
    cache     Cache[K, V]         // live | lru | unbounded
    negatives *negativeStore[K]   // nil unless enabled
    refresher KeyedRefresher[K, V]// none | preload | delta-poll (§8.9); Start()ed in New, Stop()ped on close
    metrics   Recorder
}

All slots are interfaces (KeyedSource, Transformer, Cache, KeyedRefresher), each implemented by a small reusable struct — one uniform model.

Read path Get(ctx, key) (single — account, GVL):

  1. cache.Get(key) → positive hit returns typed V (pure lookup, no unmarshal).
  2. if negatives enabled and key blocked → return NotFound (no backend call).
  3. miss → source.Fetch(ctx, []K{key}) → classify:
    • OK → transform once → cache.Save → return.
    • 404/400 (definitive) → if negatives enabled negatives.mark; return NotFound.
    • 5xx/timeout/network → return error, do NOT cache.

Read path GetBatch(ctx, keys) (list — stored req/resp):

  1. cache.GetBatch(keys) → found, missing.
  2. if missing empty → return found.
  3. source.Fetch(ctx, missing) → one batch call → transform each → cache.Save → merge into found.

Keyless engine (currency). SnapshotFetcher[V] has no loader on the read path — Get() is a pure snap.Get() atomic load; the SnapshotRefresher (periodic) is the only writer — no negatives, no key. See §8.4.

type SnapshotFetcher[V any] struct {
    snap      Snapshot[V]
    refresher SnapshotRefresher[V] // periodic; Start()ed in NewSnapshot, Stop()ped on close
}
func (f *SnapshotFetcher[V]) Get() V { return f.snap.Get() } // no fetch, no transform, no key

Herd resistance (livesite incident)

Under load, every auction for one account id hit the fetcher and got a bad request; with no coalescing (N misses ⇒ N backend calls) and failures never cached, this stampeded the backend. Following are the in-process, per-pod layers:

  • Backpressure — bounded-concurrency cap on the Source for many-distinct-key miss bursts (stdlib http.Transport.MaxConnsPerHost).
  • Jitter — TTL ±10% at Save, plus full-jitter exponential backoff on retries (hashicorp/go-retryablehttp).
account (hot path) currency / GVL (background)
Retry with backoff+jitter no, fail fast under tmax yes, no deadline pressure
TTL jitter yes, ±10% (cold-start protection) n/a (preload/periodic, not TTL)
layer dimension mechanism default
negative cache across-time repeats of a definitive per-id verdict cache 404/NotFound + per-id 400 for negTTL ON for accounts
circuit breaker on Source systemic 5xx/timeout/overload error-rate breaker; open ⇒ fail fast (stale/defaults) on for remote sources

Classification trap — negative-cache a 400 only when it is a per-id verdict ("this id is unknown/invalid"). A systemic 400 (malformed batch / API rejecting every call) feeds the breaker, never per-id negatives — else every id is poison-cached as false NotFound. This requires the fetcher contract to distinguish "bad id" from "unhealthy service"; today's HTTP fetcher collapses both and must be tightened.

Per-pod scope: all three are per-pod state. With P replicas the cross-pod floor for a hot key is P (one call per pod), not 1. Rollout/HPA brings up cold-cache pods that re-stampede — mitigate with staggered rollout, optional warmup, and the breaker. A shared tier (Redis) fronting the Source is future scope.

Authenticated Source

Today's HTTP fetcher issues unauthenticated GETs; real fetcher APIs are protected, so auth is baked into the HTTP Source as a pluggable per-request decorator:

type Authenticator interface { Apply(ctx context.Context, req *http.Request) error }

Closed, config-selected set: none | static | oauth2_cc | azure_wi | mtls. Rules:

  • Credentials from mounted-secret / env refs only (config holds only env: / file: references, never inline); on AKS prefer Workload Identity (azure_wi) so no secret lives in the cluster.

5. Metrics

These are the signals that prove the kit behaves as designed once the flag is on. Emit them per value type where it matters (label subsystem="account|gvl|stored_requests|stored_responses|currency"); keep labels low-cardinality (no account id / key in labels).

Metric Type What it proves Pass condition
cache_result{result="hit|miss|negative"} counter caching is effective hit share climbs and dominates after warm-up
cache_negative_total{reason="not_found|bad_request"} counter negative cache fires on definitive verdicts > 0 after an unknown-id request; flat across most ids
backend_fetch_result{backend,result="ok|notfound|bad_request|error"} counter Source classification is correct notfound/bad_request map to negatives; error never negative-cached
backend_fetch_duration_seconds{backend} histogram upstream latency / breaker health stable; spikes correlate with breaker opens
cache_evictions_total counter max_entries is not thrashing ~0 in steady state; sustained rise ⇒ raise the cap
cache_entries gauge resident set vs configured cap plateaus below cap; at cap + rising evictions = undersized

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status
Needs Requirements

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions