You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Refactor the Prebid Server account configuration fetch path to eliminate per-request JSON parsing, default-merging, and map-building, and to collapse concurrent cache misses into a single upstream call per pod. The new design caches an immutable, fully-derived *config.Account with negative caching and per-backend metrics.
Phase 1 is scoped to account config only implementation but provides a path for GVL and stored req/response re-structuring.
2. Background
Current State
flowchart LR
subgraph Stored[stored_requests]
S1["Fetch<br>db, http, file"] --> S2["Cache<br>raw JSON, TTL"]
S3["Refresh: poll for changes <br> + invalidate"] --> S2
end
subgraph GVL[GDPR vendor list]
G1["Fetch http"] --> G2["Cache<br>ALL versions preloaded"]
G3["Refresh: on-demand <br> rate-limited ~10 min"] --> G2
end
subgraph CUR[currency]
C1["Fetch http"] --> C2["Cache<br>rates in atomic value"]
C3["Refresh: periodic timer"] --> C2
end
subgraph ACC[account hot path]
A1["Fetch raw JSON"] --> A2["Parse + merge defaults +<br> derive EVERY REQUEST <br> no typed cache"]
end
Loading
Problems: 3 different refresh loops, GVL preloads everything (startup + ~2/3 memory), account does unmarshal+merge+derive on every request.
Inefficiency — Account Fetcher
Every OpenRTB auction served by Prebid Server resolves an account configuration via account.GetAccount (see account/account.go), which calls a chain of:
The cache stores raw merged JSON bytes. On every request — cache hit or miss — GetAccount then runs:
jsonutil.UnmarshalValid(accountJSON, &config.Account) — full JSON decode.
setDerivedConfig(account) — allocates PurposeConfigs, VendorExceptionMap, BasicEnforcementVendorsMap, etc.
Account configs are growing in size as PSP adopts modularity for Bidder Optimization, so this per-request cost scales linearly with config size × QPS.
Inefficiency — GVL Fetcher
At startup, every Prebid Server instance preloads its GDPR vendor-list cache by fetching every historical version of the Global Vendor List — for each supported spec version (2 and 3), it loops from the first list version up to the latest and downloads them all:
NewVendorListFetcher (startup)
└── for spec in {2, 3}:
└── for list = first .. latest: ← every version ever published
└── HTTP GET vendor-list-v{list}.json
└── ParseEagerly → api.VendorList
└── cacheSave (never expires, never evicted)
This has two costs:
Startup fan-out — hundreds of blocking, sequential HTTP round-trips (no parallelism) before the fetcher is ready, each with a full parse.
Unbounded memory — the cache never expires or evicts, so all versions stay resident for the life of the process, even though live traffic only references the latest few.
Both grow linearly with the total number of published GVL versions, while the actually-used working set stays roughly constant (the most recent handful). The redesign bounds preload to the latest N versions per spec, serving rare older versions lazily on demand.
flowchart LR
ID["id / ids"] --> CACHE{"Cache<br/>Get"}
CACHE -- hit --> OUT["typed value V"]
CACHE -- miss --> SRC["Source<br/>pull raw bytes"]
SRC --> TR["Transform<br/>unmarshal + <br>overlay defaults + derive"]
TR --> SAVE["Cache.Save V"]
SAVE --> OUT
REFRESH["Refresh<br/>none or preload"] -. "seeds/expires" .-> CACHE
Loading
Key rule: the cache stores the composed typed value V, not raw JSON. Transform runs once per id at insert time. Hot path on a hit = pure map lookup, no unmarshal, no json merge.
Source, Transform and Cache above are generic interfaces — the engine is generic over [K, V] and never names a concrete backend or value type:
// Source: I/O only. Returns undecoded bytes. Knows nothing about V.// BATCH-keyed — a single lookup is a one-element slice. (http_fetcher.FetchAccounts /// FetchRequests / FetchResponses are already batch; the single-key methods are the shims.)// GVL implements this too, but loops internally (one URL per version — no batch collapse).typeKeyedSource[Kcomparable] interface {
Fetch(ctx context.Context, keys []K) (map[K]json.RawMessage, []error)
}
// KeylessSource: same http transport as KeyedSource, just a fixed URL and no key (one document).// NOT a separate backend — a thin shape adapter over the same GET+read-body, used by CURRENCY only.typeKeylessSourceinterface {
Fetch(ctx context.Context) (json.RawMessage, error)
}
// Transformer: raw bytes -> typed V. Runs ONCE per value at cache insert. Uniform for all four.// (account = unmarshal+defaults+derive; GVL = ParseEagerly; currency = Rates unmarshal;// stored = Identity(), a typed no-op that returns the bytes unchanged.)typeTransformer[Vany] interface {
Transform(raw json.RawMessage) (V, error)
}
// Cache: keyed store of composed typed V. Authoritative def (Get/GetBatch/Save/SaveNegative/// Invalidate + the `state` enum). Used by account, stored, GVL.typeCache[Kcomparable, Vany] interface// Snapshot: keyless single-value box. Used by CURRENCY only. Read = pure atomic load.typeSnapshot[Vany] interface {
Get() V// current value, or fallback if never filledStore(valV)
}
// Refresher: WHEN the container is populated / kept fresh. Two families, matched to the two// containers. The only slot with a lifecycle.typeKeyedRefresher[Kcomparable, Vany] interface { // none | preload | delta-pollStart(ctx context.Context, sKeyedSource[K], tTransformer[V], cCache[K, V])
Stop()
}
typeSnapshotRefresher[Vany] interface { // periodic (currency)Start(ctx context.Context, sKeylessSource, tTransformer[V], snapSnapshot[V])
Stop()
}
Concrete backends (HTTPSource, SQLSource, FileSource, EmptyFetcher) each return a KeyedSource; currency returns a KeylessSource. The value type is chosen by the Transformer (AccountTransformer → *config.Account, GVLTransformer → api.VendorList, CurrencyTransformer → *Rates, Identity → json.RawMessage). Two constructors wire them:
The split is not cosmetic: New puts a loader on the read path (miss → fetch → transform → save with negatives); NewSnapshot is a read-only view over a box the periodic refresher fills — reads never fetch, so no negatives, no key.
The three closed axes
A design is one pick from each axis.
Source (where the bytes come from)
http — fetch raw JSON from a remote HTTP endpoint
sql — read a JSON blob column from a database
file — read JSON from a local file
empty — null-object; no backend, every lookup returns not-found
Cache (retention policy)
nil — no caching, every Get hits the Source
ttl — entries expire after a configured interval
load-once — populated once (e.g. at startup), never expires
negative decorator — optionally layered on any of the above to cache definitive not-found verdicts
snapshot — keyless single-value box, overwritten wholesale on refresh
Refresh (freshness)
ttl — freshness via TTL expiry only
none — never refresh
preload — warm the cache up front
delta-poll — future
periodic — timer re-fetches the whole value and overwrites (currency; also GVL's rate-limited refresh)
Read-path — Account (TTL-only)
sequenceDiagram
participant Auc as Auction (GetAccount)
participant K as cachekit(string,*Account)
participant C as TTL+negative cache
participant S as Source (http/sql)
participant T as Transform (parseAndDerive)
Auc->>K: Get(accountID)
K->>C: lookup
alt positive hit (not expired)
C-->>K: *Account
else negative hit (unknown id, not expired)
C-->>K: NotFound (no backend call)
else miss / expired
K->>S: fetch raw JSON
alt found
S-->>T: raw
T-->>K: *Account (defaults overlaid + derived)
K->>C: Save(id, *Account, ttl)
else 404 / 400 (definitive)
S-->>K: NotFound
K->>C: Save negative (negTTL)
else 5xx / timeout / network
S-->>K: error (DO NOT cache)
end
end
K-->>Auc: *Account or NotFound
Loading
Notes:
Freshness = TTL expiry only. TTL value sits in the 3h min / 6–12h target band.
Negative cache policy from POC: cache only 404/NotFound + 400; never 401/403/429/5xx/timeout/decode.
Direct-DB companies = swap Cache to nil → every Get hits Source live (no other change).
Read-path — GVL (preload latest N, fixed N)
sequenceDiagram
participant Init as Startup
participant K as cachekit(gvlKey, VendorList)
participant C as load-once cache
participant S as gvl http Source
Init->>S: fetch latest version number
loop latest-N .. latest
Init->>S: fetch version
S-->>C: Save(specVer,listVer to VendorList) [never expires]
end
Note over C: only N recent versions resident (not all)
par at request time
K->>C: Get(specVer,listVer)
alt in cache
C-->>K: VendorList
else older than N (rare)
K->>S: on-demand fetch (future: rate-limited)
S-->>C: Save
C-->>K: VendorList
end
end
Loading
Note: Startup fetches N versions instead of all; memory drops from "all versions" to N.
Startup wiring (composition, builder-style)
flowchart TB
CFG["config: source <br> type, cache type, <br> ttl, N, refresh"] --> BUILD["cachekit.New"]
PICKS["pick Source impl<br/>pick Cache impl<br/>attach Transform<br/>pick Refresh"]
BUILD --> PICKS
PICKS --> FET["returns Source-like fetcher<br/>caller sees only Get"]
FET --> ACC["account.GetAccount"]
FET --> GVL["gdpr permissions"]
FET --> STORED["stored_requests"]
Loading
Illustrative construction (pseudo-code):
accountFetcher:=cachekit.New[string, *config.Account](
cachekit.HTTPSource(client, accountsURL, auth),
AccountTransformer{defaults: cfg.AccountDefaultsJSON()},
cachekit.LRU(maxEntries, ttl).WithNegative(negMax, negTTL),
cachekit.NoRefresh(),
)
// read: acct, err := accountFetcher.Get(ctx, "pub-42")storedResponseFetcher:=cachekit.New[string, json.RawMessage](
cachekit.SQLSource(db, respQueryTemplate),
cachekit.Identity(),
cachekit.LRU(maxEntries, ttl),
cachekit.DeltaPoll(interval, changeSource),
)
// read: m, errs := storedResponseFetcher.GetBatch(ctx, ids) // batch// (stored request+imp share one source that fills two caches; same shape)gvlFetcher:=cachekit.New[gvlKey, api.VendorList](
cachekit.HTTPSource(client, gvlURLMaker), // one URL per version (loops)GVLTransformer{},
cachekit.LoadOnce(),
cachekit.Preload(n),
)
// read: list, err := gvlFetcher.Get(ctx, gvlKey{spec, ver})// Currency — keyless: snapshot box, periodic timer.currencyFetcher:=cachekit.NewSnapshot[*currency.Rates](
cachekit.HTTPDocSource(client, syncSourceURL), // same http source, fixed URL, no keyCurrencyTransformer{},
cachekit.Box(currency.NewConstantRates(), staleWindow), // atomic value + constant-rates fallbackcachekit.Periodic(refreshInterval),
)
// read: rates := currencyFetcher.Get()// rate, err := rates.GetRate("USD", "EUR") // unchanged on top
Type mapping
Subsystem
Key
Source
Transform (compose)
Cache
Refresh
Read
Account
string
http/sql
unmarshal+defaults+derive
ttl+negative
none
Get
GVL
(spec,list)
gvl-http
ParseEagerly → VendorList
load-once
preload
Get
Stored req/imp/resp
string
sql/http
Identity (raw JSON)
ttl
delta-poll
GetBatch
Currency
(keyless)
http (no key)
Rates unmarshal
snapshot (box)
periodic
Get()
4. Low-Level Design
Cache type is selectable per fetcher via config; default is lru. There is exactly one Cache[K,V] interface and the Fetcher / callers are identical regardless of policy.
Note — v2_enabled is the feature flag that will need to be enabled to switch traffic/workflow to fetchers 2.0. By default, it will be set to false.
type
Behavior
Use case
none
always fetch from source, retain nothing
direct-DB tenants needing per-request fresh
lru
bounded read-through (DEFAULT)
account, GVL, stored-data via http/db
unbounded
authoritative mirror, no eviction
stored-data poll-mirror only
Data structures
Every cached value is the typed deserialization of a JSON source; V is chosen per subsystem:
subsystem
JSON source
cached V
Transform
Account
account JSON
*config.Account
unmarshal + merge defaults + derive (once)
GVL
vendor-list JSON
api.VendorList
ParseEagerly (once)
Stored req/imp/resp
stored JSON
json.RawMessage
identity
Note: Stored request stays bytes on purpose — it's MergePatched against the live request per auction, so there is no stable typed value to cache (same split as prebid-server-java: typed Account, raw-JSON stored data).
typestateuint8const (
stEmptystate=iota// in neither store (never seen / evicted / expired)stPositive// found in the positive storestNegative// found in the negative store (negatives enabled only)
)
// positive store value: the composed TYPED value + its expirytypeentry[Vany] struct {
vV// composed TYPED valueexpires time.Time// zero == never expires (load-once / mirror mode)
}
typeCache[Kcomparable, Vany] interface {
Get(keyK) (vV, ststate) // st derived from which store answered (account, GVL)GetBatch(keys []K) (foundmap[K]V, missing []K) // stored req/resp — partition hits vs missesSave(keyK, vV, ttl time.Duration) // ttl==0 ⇒ never expires → positive storeSaveNegative(keyK, ttl time.Duration) // → negative storeInvalidate(keyK)
}
// Keyless container for currency (§8.4). Not a Cache — no keys, no negatives, no miss path.typeSnapshot[Vany] interface {
Get() V// current value, or constant-rates fallback if never filledStore(valV) // written only by the periodic refresher
}
Read: Get checks the positive store first (→ stPositive), then the negative store (→ stNegative), else stEmpty.
Worked example (account cache, negatives ON)
Four sequential Get calls; the two stores shown after each. state is recomputed each call from which store answered — it is never persisted.
#
call
actual value cached?
negative value cached?
state
action taken
1
Get("pub-123") (real)
no
no
stEmpty
fetch → 200 → Save("pub-123", acct, 1h)
2
Get("pub-123") again
yes
—
stPositive
return acct — no backend, no unmarshal
3
Get("pub-999") (absent)
no
no
stEmpty
fetch → 404 → SaveNegative("pub-999", 60s)
4
Get("pub-999") again
no
yes
stNegative
return NotFound — no backend call
Three impls: live (no-op), lru (default — thin wrapper over hashicorp/golang-lru/v2expirable.LRU), unbounded (map + RWMutex, poll-mirror only). Negatives are a separate bounded store (own small cap + short negTTL), so an unknown-key flood can never evict real data; instantiated only when negative.enabled: true. The reason for using hashicorp/golang-lru/v2 vs existing coocood/freecache is that the latter doesn't support typed values to be cached.
The Fetcher
typeFetcher[Kcomparable, Vany] struct {
sourceKeyedSource[K] // BATCH raw fetch (http/db/file/empty); single = one-element slicetransformTransformer[V] // bytes → typed V, ONCE at insertcacheCache[K, V] // live | lru | unboundednegatives*negativeStore[K] // nil unless enabledrefresherKeyedRefresher[K, V]// none | preload | delta-poll (§8.9); Start()ed in New, Stop()ped on closemetricsRecorder
}
All slots are interfaces (KeyedSource, Transformer, Cache, KeyedRefresher), each implemented by a small reusable struct — one uniform model.
Read path Get(ctx, key) (single — account, GVL):
cache.Get(key) → positive hit returns typed V (pure lookup, no unmarshal).
if negatives enabled and key blocked → return NotFound (no backend call).
miss → source.Fetch(ctx, []K{key}) → classify:
OK → transform once → cache.Save → return.
404/400 (definitive) → if negatives enabled negatives.mark; return NotFound.
source.Fetch(ctx, missing) → one batch call → transform each → cache.Save → merge into found.
Keyless engine (currency). SnapshotFetcher[V] has no loader on the read path — Get() is a pure snap.Get() atomic load; the SnapshotRefresher (periodic) is the only writer — no negatives, no key. See §8.4.
typeSnapshotFetcher[Vany] struct {
snapSnapshot[V]
refresherSnapshotRefresher[V] // periodic; Start()ed in NewSnapshot, Stop()ped on close
}
func (f*SnapshotFetcher[V]) Get() V { returnf.snap.Get() } // no fetch, no transform, no key
Herd resistance (livesite incident)
Under load, every auction for one account id hit the fetcher and got a bad request; with no coalescing (N misses ⇒ N backend calls) and failures never cached, this stampeded the backend. Following are the in-process, per-pod layers:
Backpressure — bounded-concurrency cap on the Source for many-distinct-key miss bursts (stdlib http.Transport.MaxConnsPerHost).
Jitter — TTL ±10% at Save, plus full-jitter exponential backoff on retries (hashicorp/go-retryablehttp).
account (hot path)
currency / GVL (background)
Retry with backoff+jitter
no, fail fast under tmax
yes, no deadline pressure
TTL jitter
yes, ±10% (cold-start protection)
n/a (preload/periodic, not TTL)
layer
dimension
mechanism
default
negative cache
across-time repeats of a definitive per-id verdict
cache 404/NotFound + per-id 400 for negTTL
ON for accounts
circuit breaker on Source
systemic 5xx/timeout/overload
error-rate breaker; open ⇒ fail fast (stale/defaults)
on for remote sources
Classification trap — negative-cache a 400 only when it is a per-id verdict ("this id is unknown/invalid"). A systemic 400 (malformed batch / API rejecting every call) feeds the breaker, never per-id negatives — else every id is poison-cached as false NotFound. This requires the fetcher contract to distinguish "bad id" from "unhealthy service"; today's HTTP fetcher collapses both and must be tightened.
Per-pod scope: all three are per-pod state. With P replicas the cross-pod floor for a hot key is P (one call per pod), not 1. Rollout/HPA brings up cold-cache pods that re-stampede — mitigate with staggered rollout, optional warmup, and the breaker. A shared tier (Redis) fronting the Source is future scope.
Authenticated Source
Today's HTTP fetcher issues unauthenticated GETs; real fetcher APIs are protected, so auth is baked into the HTTP Source as a pluggable per-request decorator:
Credentials from mounted-secret / env refs only (config holds only env: / file: references, never inline); on AKS prefer Workload Identity (azure_wi) so no secret lives in the cluster.
5. Metrics
These are the signals that prove the kit behaves as designed once the flag is on. Emit them per value type where it matters (label subsystem="account|gvl|stored_requests|stored_responses|currency"); keep labels low-cardinality (no account id / key in labels).
Config Fetcher — Refactor (Phase 1)
1. Summary
Refactor the Prebid Server account configuration fetch path to eliminate per-request JSON parsing, default-merging, and map-building, and to collapse concurrent cache misses into a single upstream call per pod. The new design caches an immutable, fully-derived
*config.Accountwith negative caching and per-backend metrics.Phase 1 is scoped to account config only implementation but provides a path for GVL and stored req/response re-structuring.
2. Background
Current State
flowchart LR subgraph Stored[stored_requests] S1["Fetch<br>db, http, file"] --> S2["Cache<br>raw JSON, TTL"] S3["Refresh: poll for changes <br> + invalidate"] --> S2 end subgraph GVL[GDPR vendor list] G1["Fetch http"] --> G2["Cache<br>ALL versions preloaded"] G3["Refresh: on-demand <br> rate-limited ~10 min"] --> G2 end subgraph CUR[currency] C1["Fetch http"] --> C2["Cache<br>rates in atomic value"] C3["Refresh: periodic timer"] --> C2 end subgraph ACC[account hot path] A1["Fetch raw JSON"] --> A2["Parse + merge defaults +<br> derive EVERY REQUEST <br> no typed cache"] endProblems: 3 different refresh loops, GVL preloads everything (startup + ~2/3 memory), account does unmarshal+merge+derive on every request.
Inefficiency — Account Fetcher
Every OpenRTB auction served by Prebid Server resolves an account configuration via
account.GetAccount(seeaccount/account.go), which calls a chain of:The cache stores raw merged JSON bytes. On every request — cache hit or miss —
GetAccountthen runs:jsonutil.UnmarshalValid(accountJSON, &config.Account)— full JSON decode.setDerivedConfig(account)— allocatesPurposeConfigs,VendorExceptionMap,BasicEnforcementVendorsMap, etc.Account configs are growing in size as PSP adopts modularity for Bidder Optimization, so this per-request cost scales linearly with config size × QPS.
Inefficiency — GVL Fetcher
At startup, every Prebid Server instance preloads its GDPR vendor-list cache by fetching every historical version of the Global Vendor List — for each supported spec version (2 and 3), it loops from the first list version up to the latest and downloads them all:
This has two costs:
Both grow linearly with the total number of published GVL versions, while the actually-used working set stays roughly constant (the most recent handful). The redesign bounds preload to the latest N versions per spec, serving rare older versions lazily on demand.
3. Proposed model — Source × Cache × Transform (+ optional Refresh)
flowchart LR ID["id / ids"] --> CACHE{"Cache<br/>Get"} CACHE -- hit --> OUT["typed value V"] CACHE -- miss --> SRC["Source<br/>pull raw bytes"] SRC --> TR["Transform<br/>unmarshal + <br>overlay defaults + derive"] TR --> SAVE["Cache.Save V"] SAVE --> OUT REFRESH["Refresh<br/>none or preload"] -. "seeds/expires" .-> CACHEKey rule: the cache stores the composed typed value
V, not raw JSON. Transform runs once per id at insert time. Hot path on a hit = pure map lookup, no unmarshal, no json merge.Source, Transform and Cache above are generic interfaces — the engine is generic over
[K, V]and never names a concrete backend or value type:Concrete backends (
HTTPSource,SQLSource,FileSource,EmptyFetcher) each return aKeyedSource; currency returns aKeylessSource. The value type is chosen by the Transformer (AccountTransformer→*config.Account,GVLTransformer→api.VendorList,CurrencyTransformer→*Rates,Identity→json.RawMessage). Two constructors wire them:New[K, V](KeyedSource[K], Transformer[V], Cache[K, V], KeyedRefresher[K, V])→ account, stored, GVL.NewSnapshot[V](KeylessSource, Transformer[V], Snapshot[V], SnapshotRefresher[V])→ currency.The split is not cosmetic:
Newputs a loader on the read path (miss → fetch → transform → save with negatives);NewSnapshotis a read-only view over a box the periodic refresher fills — reads never fetch, so no negatives, no key.The three closed axes
A design is one pick from each axis.
http— fetch raw JSON from a remote HTTP endpointsql— read a JSON blob column from a databasefile— read JSON from a local fileempty— null-object; no backend, every lookup returns not-foundnil— no caching, every Get hits the Sourcettl— entries expire after a configured intervalload-once— populated once (e.g. at startup), never expiresnegative decorator— optionally layered on any of the above to cache definitive not-found verdictssnapshot— keyless single-value box, overwritten wholesale on refreshttl— freshness via TTL expiry onlynone— never refreshpreload— warm the cache up frontdelta-poll— futureperiodic— timer re-fetches the whole value and overwrites (currency; also GVL's rate-limited refresh)Read-path — Account (TTL-only)
sequenceDiagram participant Auc as Auction (GetAccount) participant K as cachekit(string,*Account) participant C as TTL+negative cache participant S as Source (http/sql) participant T as Transform (parseAndDerive) Auc->>K: Get(accountID) K->>C: lookup alt positive hit (not expired) C-->>K: *Account else negative hit (unknown id, not expired) C-->>K: NotFound (no backend call) else miss / expired K->>S: fetch raw JSON alt found S-->>T: raw T-->>K: *Account (defaults overlaid + derived) K->>C: Save(id, *Account, ttl) else 404 / 400 (definitive) S-->>K: NotFound K->>C: Save negative (negTTL) else 5xx / timeout / network S-->>K: error (DO NOT cache) end end K-->>Auc: *Account or NotFoundNotes:
nil→ every Get hits Source live (no other change).Read-path — GVL (preload latest N, fixed N)
sequenceDiagram participant Init as Startup participant K as cachekit(gvlKey, VendorList) participant C as load-once cache participant S as gvl http Source Init->>S: fetch latest version number loop latest-N .. latest Init->>S: fetch version S-->>C: Save(specVer,listVer to VendorList) [never expires] end Note over C: only N recent versions resident (not all) par at request time K->>C: Get(specVer,listVer) alt in cache C-->>K: VendorList else older than N (rare) K->>S: on-demand fetch (future: rate-limited) S-->>C: Save C-->>K: VendorList end endNote: Startup fetches N versions instead of all; memory drops from "all versions" to N.
Startup wiring (composition, builder-style)
flowchart TB CFG["config: source <br> type, cache type, <br> ttl, N, refresh"] --> BUILD["cachekit.New"] PICKS["pick Source impl<br/>pick Cache impl<br/>attach Transform<br/>pick Refresh"] BUILD --> PICKS PICKS --> FET["returns Source-like fetcher<br/>caller sees only Get"] FET --> ACC["account.GetAccount"] FET --> GVL["gdpr permissions"] FET --> STORED["stored_requests"]Illustrative construction (pseudo-code):
Type mapping
stringGet(spec,list)VendorListGetstringGetBatchGet()4. Low-Level Design
Cache type is selectable per fetcher via config; default is
lru. There is exactly oneCache[K,V]interface and the Fetcher / callers are identical regardless of policy.v2_enabledis the feature flag that will need to be enabled to switch traffic/workflow to fetchers 2.0. By default, it will be set tofalse.nonelruunboundedData structures
Every cached value is the typed deserialization of a JSON source;
Vis chosen per subsystem:V*config.Accountapi.VendorListjson.RawMessageNote: Stored request stays bytes on purpose — it's MergePatched against the live request per auction, so there is no stable typed value to cache (same split as prebid-server-java: typed Account, raw-JSON stored data).
Read:
Getchecks the positive store first (→stPositive), then the negative store (→stNegative), elsestEmpty.Worked example (account cache, negatives ON)
Four sequential
Getcalls; the two stores shown after each.stateis recomputed each call from which store answered — it is never persisted.Get("pub-123")(real)stEmptySave("pub-123", acct, 1h)Get("pub-123")againstPositiveGet("pub-999")(absent)stEmptySaveNegative("pub-999", 60s)Get("pub-999")againstNegativeThree impls:
live(no-op),lru(default — thin wrapper overhashicorp/golang-lru/v2expirable.LRU),unbounded(map + RWMutex, poll-mirror only). Negatives are a separate bounded store (own small cap + short negTTL), so an unknown-key flood can never evict real data; instantiated only whennegative.enabled: true. The reason for usinghashicorp/golang-lru/v2vs existingcoocood/freecacheis that the latter doesn't support typed values to be cached.The Fetcher
All slots are interfaces (
KeyedSource,Transformer,Cache,KeyedRefresher), each implemented by a small reusable struct — one uniform model.Read path
Get(ctx, key)(single — account, GVL):cache.Get(key)→ positive hit returns typedV(pure lookup, no unmarshal).source.Fetch(ctx, []K{key})→ classify:cache.Save→ return.negatives.mark; return NotFound.Read path
GetBatch(ctx, keys)(list — stored req/resp):cache.GetBatch(keys)→ found, missing.source.Fetch(ctx, missing)→ one batch call → transform each →cache.Save→ merge into found.Keyless engine (currency).
SnapshotFetcher[V]has no loader on the read path —Get()is a puresnap.Get()atomic load; theSnapshotRefresher(periodic) is the only writer — no negatives, no key. See §8.4.Herd resistance (livesite incident)
Under load, every auction for one account id hit the fetcher and got a bad request; with no coalescing (N misses ⇒ N backend calls) and failures never cached, this stampeded the backend. Following are the in-process, per-pod layers:
http.Transport.MaxConnsPerHost).hashicorp/go-retryablehttp).Classification trap — negative-cache a 400 only when it is a per-id verdict ("this id is unknown/invalid"). A systemic 400 (malformed batch / API rejecting every call) feeds the breaker, never per-id negatives — else every id is poison-cached as false NotFound. This requires the fetcher contract to distinguish "bad id" from "unhealthy service"; today's HTTP fetcher collapses both and must be tightened.
Per-pod scope: all three are per-pod state. With P replicas the cross-pod floor for a hot key is P (one call per pod), not 1. Rollout/HPA brings up cold-cache pods that re-stampede — mitigate with staggered rollout, optional warmup, and the breaker. A shared tier (Redis) fronting the Source is future scope.
Authenticated Source
Today's HTTP fetcher issues unauthenticated GETs; real fetcher APIs are protected, so auth is baked into the HTTP Source as a pluggable per-request decorator:
Closed, config-selected set:
none | static | oauth2_cc | azure_wi | mtls. Rules:env:/file:references, never inline); on AKS prefer Workload Identity (azure_wi) so no secret lives in the cluster.5. Metrics
These are the signals that prove the kit behaves as designed once the flag is on. Emit them per value type where it matters (label
subsystem="account|gvl|stored_requests|stored_responses|currency"); keep labels low-cardinality (no account id / key in labels).cache_result{result="hit|miss|negative"}cache_negative_total{reason="not_found|bad_request"}backend_fetch_result{backend,result="ok|notfound|bad_request|error"}backend_fetch_duration_seconds{backend}cache_evictions_totalcache_entries