Skip to content

Commit c058a57

Browse files
prk-Jrjevansnycclaudearam356
authored
Implement server-side ad slot templates with PBS and APS auction (#680)
* Add server-side ad templates design spec Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Update server-side ad templates spec and add implementation plan - Incorporate all review feedback (aram356 + jevansnyc): cache contract, consent/GDPR gating, async restructuring detail, CreativeOpportunityFormat schema, glob pattern fix, XSS escaping, win notifications, APS params, timeout config key, defineSlot fix, gpt.rs ownership, KV migration path, Phase 2 sketch - Fix Prettier formatting (format-docs CI) - Add implementation plan (12 tasks, TDD, ordered by dependency) * Update server-side ad templates spec and add implementation plan - Incorporate all review feedback (aram356 + jevansnyc): cache contract, consent/GDPR gating, async restructuring detail, CreativeOpportunityFormat schema, glob pattern fix, XSS escaping, win notifications, APS params, timeout config key, defineSlot fix, gpt.rs ownership, KV migration path, Phase 2 sketch - Fix Prettier formatting (format-docs CI) - Add implementation plan (12 tasks, TDD, ordered by dependency) * Update rust edition from 2021 to 2024 * Rework spec for non-blocking page rendering with /ts-bids fetch Replace the head-injected __ts_bids design with a server-cached bid delivery model fetched by the client via a new /ts-bids endpoint. The auction never blocks page rendering — </head> flushes immediately, body parses without waiting for bids, and the client fetches bids in parallel with content paint. Key changes: - §2 Goal: bid delivery decoupled from page rendering; FCP unchanged from no-TS baseline - §4.3 Auction Trigger: drop buffered/streaming dichotomy; single mode forces chunked encoding on all origins (WordPress, NextJS, etc.) - §4.4 Head Injection: only __ts_ad_slots and __ts_request_id injected at <head> open; bid results moved to /ts-bids endpoint - §4.6 Client Residual: __tsAdInit defines slots immediately, fetches bids via /ts-bids, applies targeting and fires refresh() after resolve - §4.7 (new) Caching Behavior: explicit cacheability table for HTML, JS, CSS, tsjs bundle, bid results; Fastly edge HTTP cache leveraged for origin HTML - §5 Request-Time Sequence: full mermaid diagram covering content + creative + burl flow with cache-hit and cache-miss branches; separate text sequences for cache-hit (~80ms FCP, ~900ms ad-visible) and cache-miss (~250ms FCP, ~1,050ms ad-visible) - §6 Performance Summary: cache-hit and cache-miss columns; FCP added as a tracked metric - §7 Implementation Scope: add bid_cache.rs, /ts-bids endpoint, force chunked encoding step - §8 Edge Cases: origin-agnostic entries; new entries for /ts-bids 404 and client-never-fetches-/ts-bids Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fmt docs and update plan as per spec * Rework ad templates spec for body injection and add identity model Pivot from the /ts-bids fetch endpoint + in-process bid_cache design to inline __ts_bids injection before </body>. The earlier design relied on shared state that doesn't reliably survive Fastly Compute's per-request Wasm isolate model — body injection achieves the same FCP property in a single response with no shared-state requirement. Key changes: - §4.3: replace /ts-bids long-poll with bounded </body> hold tied to A_deadline. Body content above </body> paints first; close-tag held until auction completes or A_deadline fires (graceful __ts_bids = {} fallback). - §4.3: add auction-eligibility gating (consent, bot UA, prefetch hints, HEAD method, slot match) so auctions fire on real first-page-load impressions only. - §4.4: replace __ts_request_id + /ts-bids machinery with two inline <script> blocks — __ts_ad_slots at <head> open, __ts_bids before </body> via lol_html el.on_end_tag(). - §4.5: move both nurl and burl to client-side firing from slotRenderEnded after hb_adid match. Server-side firing rejected to avoid billing inflation on bids that never render. - §4.6: replace fetch+Promise pattern with synchronous __ts_bids read. Add lazy slim-Prebid loader (post-window.load) for scroll/refresh auctions and Phase B identity warm-up. Add ts_initial=1 slot-ownership sentinel. - §4.7: switch Cache-Control from private, no-store to private, max-age=0 to preserve browser BFCache eligibility while still preventing intermediate-cache leaks. - §4.8 (new): document the EC/KV identity model as load-bearing auction input — Phase A retrieval at request time, Phase B post-render enrichment via slim-Prebid userID modules. Add bare-EC first-impression caveat and auction_eid_count metric. Note federated-consortium passphrase property and clickstream-compounding speed win. - §5: update mermaid + cache-hit/miss timelines for bounded body hold; ad-visible converges to ~870ms (hit) / ~1,020ms (miss). - §6: drop /ts-bids RTT row; add DCL row; add clickstream-compounding, TS-overhead, identity-coverage, and confidence-interval framing. - §7: drop bid_cache.rs and /ts-bids endpoint from scope; add auction-eligibility gating and slim-Prebid bundle build target. Add explicit "Deleted" subsection. - §8: drop /ts-bids edge cases; add SPA/pushState, bare-EC, bot/prefetch, HEAD, BFCache restoration cases. - §9.6: server-side GAM downgraded from "Phase 2 commitment" to aspirational and contingent on Google agreement. §9.8 (slim-Prebid bundle composition), §9.9 (Privacy Sandbox), §9.10 (per-bidder consent) added as follow-ups. Implementation plan at docs/superpowers/plans/2026-04-30-server-side-ad-templates.md is now stale relative to this spec; needs regenerating before code lands. * Update the server side ad template implementation plan * Add glob workspace dependency for URL pattern matching * Add Prebid price granularity bucketing (dense default, auto = dense) * Add MediaType::banner(), Bid::ad_id, and suppress_nurl config to PrebidConfig * Add creative_opportunities config types, URL glob matching, APS params wiring * Wire CreativeOpportunitiesConfig into Settings; add creative-opportunities.toml Adds the creative_opportunities field to Settings struct to deserialize configuration for the server-side ad auction feature. Includes build.rs stubs for types required during build-time configuration validation. Creates creative-opportunities.toml with example slot configuration and updates trusted-server.toml with the [creative_opportunities] section defining GAM network ID, auction timeout, and price granularity settings. Tests pass with proper TOML parsing of the creative_opportunities section. * Validate creative-opportunities.toml slot IDs at build time using inline TOML parse * Inject __ts_ad_slots at head-open and __ts_bids before </body> via shared auction state - Add `ad_slots_script: Option<String>` and `ad_bids_state: Arc<RwLock<Option<String>>>` fields to `HtmlProcessorConfig` - Update `from_settings` to initialize both new fields with safe defaults - Prepend `ad_slots_script` inside the existing `<head>` handler before integration inserts - Add `element!("body", ...)` handler that uses `end_tag_handlers()` to inject `__ts_bids` before `</body>`; falls back to empty `{}` when auction state is `None` - Add `IntegrationRegistry::empty_for_tests()` test helper - Add three new tests covering all injection paths * Convert handle_publisher_request to async; body-inject __ts_bids; eligibility gates; max-age=0 - Make handle_publisher_request async; add orchestrator and slots_file params - Dispatch origin request with send_async before running auction in parallel - Gate auction on GET, no prefetch, no bot, matched slots, TCF purpose-1 consent - Run server-side auction and write bucketed bids to ad_bids_state Arc<RwLock> - Compute ad_slots_script after response headers; set Cache-Control: private, max-age=0 - Fix Stream arm to thread actual ad_slots_script and ad_bids_state through - Add build_auction_request, build_bid_map, build_bids_script, build_ad_slots_script helpers - Update route_tests.rs to pass empty slots_file to route_request * Emit __tsAdInit with synchronous window.__ts_bids read; nurl+burl from slotRenderEnded * Fix bid map shape and ad slots property names; resolve clippy errors - build_bid_map now returns serde_json::Map with full bid objects (hb_pb, hb_bidder, hb_adid, nurl, burl) instead of a plain CPM string map - build_bids_script / build_ad_slots_script now emit full <script> tags using JSON.parse("…") for safe inline embedding; add html_escape_for_script helper - build_ad_slots_script uses correct property names (gam_unit_path, div_id, formats, targeting) matching the client-side TSJS bundle expectations - Replace map_or(false, …) with is_some_and(…) on lines 546, 549, 567 - Add # Panics doc sections to handle_publisher_request and create_html_processor * Wire slots_file and orchestrator into adapter; parse creative-opportunities.toml at startup * Add synchronous __tsAdInit (reads window.__ts_bids inline); nurl+burl from slotRenderEnded; slim-Prebid lazy loader * Fix cmd type regression and null guard in installTsAdInit * Add end-to-end publisher helper tests for body-injection architecture * Enable server-side auction with APS provider and adserver_mock mediator - Enable APS and adserver_mock in auction config; set providers and mediator - Increase auction_timeout_ms from 500ms to 3000ms — 500ms was too tight for HTTPS round-trips to mocktioneer, leaving the mediator zero budget - Fix mediation request: send numeric price instead of opaque encoded_price; mocktioneer requires a decoded price field and does not support encoded_price - Expand creative-opportunities slot page_patterns to include /news/** * Fix adserver_mock test for numeric price; fix GPT JS formatting * Replace explicit any in GPT integration with typed interfaces Define SlotRenderEndedEvent, SlotRenderEvent, and TestWindow types to eliminate all @typescript-eslint/no-explicit-any violations in gpt/index.ts and gpt/index.test.ts. Extend GptWindow with __tsjs_slim_prebid_url so installSlimPrebidLoader avoids the any cast. * Update creative-opportunities config to real autoblog.com GAM values Set gam_network_id to 88059007 (autoblog production network). Update atf_sidebar_ad slot to /88059007/autoblog/news with div_id ad-atf_sidebar-0-_r_2_ (desktop ATF sidebar, 300x250); restrict page_patterns to article paths only (/20**, /news/**) since that div does not exist on the homepage. Add homepage_header_ad slot targeting /88059007/autoblog/homepage with ad-header-0-_R_jpalubtak5lb_ for 970x90/728x90/970x250 leaderboard formats. Reduce auction_timeout_ms from 3000 to 500 to cap TTFB at the spec-recommended ceiling. * Update auction timeout and APS slot ID bug * Call __tsAdInit after injecting __ts_bids into page The bids script set window.__ts_bids but never invoked the __tsAdInit function, leaving GPT slots undefined and server-side targeting (hb_pb, hb_bidder) never applied. Both the winning-bid path (build_bids_script) and the no-auction fallback (html_processor None branch) now guard-call the function after the assignment. * Fix format error * Add PBS inline bidder params via creative-opportunities.toml Adds [slot.providers.pbs.bidders] support so PBS bidder params live in creative-opportunities.toml alongside APS params, without needing PBS stored requests configured server-side. PrebidAuctionProvider now sends imp.ext.prebid.storedrequest.id as a fallback for slots with no inline PBS params, and skips non-PBS provider keys (e.g. "aps") that belong to separate auction providers. PrebidImpExt gains an optional storedrequest field; empty bidder maps are omitted during serialisation. Wires mocktioneer and criteo (placeholder IDs) for both autoblog creative-opportunity slots. * Fix clippy errors * Fix test assertion * Fix double __ts_bids injection * Fix max-age cookie issue -> no-store * Add /__ts/page-bids endpoint for pushState/replaceState * Fix format ts * __tsDivToSlotId now replaced per navigation (not merged) — stale div_id entries from destroyed slots no longer persist * Update timeout for mocktioneer * Revert with updated tiomeout * Wip: Align with the spec * Fix clippy explicit-auto-deref in stream_publisher_body_async call * Fix Cache-Control headers applied only when slots matched — apply to all HTML responses per spec §4.7 + §8 * cargo fmt * Fix auction consent gate blocking non-GDPR regions — only require TCF Purpose 1 when gdpr_applies * Fix SSP requests using placeholder headers — pass real request to dispatch_auction dispatch_auction was building AuctionContext with a placeholder Request (GET https://placeholder.invalid/) that carried no headers. Prebid's request_bids copies User-Agent, x-forwarded-for, Referer, Accept-Language, and cookies from context.request before sending to Prebid Server, so SSPs received stripped requests and returned empty bids. Fix: dispatch SSP requests before req.send_async(), using the original request directly as AuctionContext.request. DispatchedAuction holds no lifetime reference to Request, so the borrow ends at return and req can be modified (restrict_accept_encoding, Host header) and sent to origin immediately after. * Fix async auction collect abandoning SSP bids when origin is slow In collect_dispatched_auction, the select loop checked `auction_start.elapsed() >= deadline` after each SSP response and broke early if the 1500ms budget had elapsed. When origin TTFB + body download exceeded the auction budget, the check fired after collecting the first SSP response, abandoning the second SSP's already-buffered response. This left responses with only one (possibly errored) SSP, causing remaining_ms == 0 which skipped the mediator, and select_winning_bids on the partial set returned zero bids. The deadline break is wrong in this context: SSP HTTP connections are already bounded by the backend first_byte_timeout set at dispatch time (1000ms per provider). By the time collect is called at origin EOF, all SSPs have either responded or been errored by Fastly's host. The select() calls drain instantly — no WASM-level deadline enforcement is needed or safe. Also add info-level log statements at dispatch, collect, and write_bids_to_state to make the auction pipeline observable without requiring a dashboard. * Cargo fmt * Fix mediator always skipped when origin body exceeds SSP auction budget In collect_dispatched_auction, the mediator was skipped when remaining_budget_ms(auction_start, timeout_ms) == 0. In the async-dispatch path, auction_start is set before pending_origin.wait(), so elapsed time includes the full origin TTFB and body download. For heavy SSR pages (autoblog), this exceeds the 1500ms SSP budget, making remaining_ms == 0 at every collection and causing the mediator to be permanently skipped. The mediator (adserver_mock) is the primary bid source — SSPs alone return no bids. Skipping it means window.__ts_bids == {} on every full page load, while handle_page_bids (which uses the sequential run_auction path) works correctly because it measures remaining time from after SSP collection. Fix: give the mediator its own configured timeout (mediator.timeout_ms()) instead of the exhausted SSP budget. This mirrors how run_parallel_mediation works: the mediator's deadline is independent of SSP round-trip time. Side effect: mediator backend name is now stable (always t1000 for adserver_mock) rather than varying per request with remaining_ms. * Cargo fmt * Adding debug info for auction * Fix auction bids missing on Next.js buffered HTML path `classify_response_route` returns `BufferedProcessed` when HTML has post-processors registered (e.g. the Next.js integration registers one via `with_html_post_processor`). Unlike the `Stream` path, which drives `one_behind_loop` to collect the dispatched auction at origin EOF, the `BufferedProcessed` branch previously discarded `dispatched_auction` entirely — so `ad_bids_state` stayed `None` and lol_html injected the fallback `window.__ts_bids = {}` instead of real bids. Fix: collect the in-flight dispatched auction in the `BufferedProcessed` branch before calling `process_response_streaming`, using the same `collect_dispatched_auction` + `write_bids_to_state` pattern that the stream path uses. The `debug.auction_html_comment` injection is mirrored here as well so the comment appears in both code paths when enabled. * Add path label and auction time to debug HTML comment * Fix XSS in script injection and cap mediator at A_deadline html_escape_for_script now unicode-escapes <, >, & and U+2028/2029 in addition to \ and ". These characters allow a crafted bid value to break out of the <script> block or terminate the JS string in some parsers. AuctionOrchestrator::collect_dispatched_auction now computes remaining budget (A_deadline - elapsed) before invoking the mediator. If the budget is already exhausted the mediator is skipped and SSP bids are returned directly; otherwise the mediator timeout is capped at the tighter of its configured value and the remaining budget, preventing it from running past A_deadline. * Added footer slot id * Fix page-bids auction context, protect Cache-Control from operator override Pass the real incoming request to AuctionContext in handle_page_bids instead of a placeholder — SSPs now receive browser UA, referer, and cookies on SPA navigation bids. Guard Cache-Control in finalize_response so operator response_headers cannot overwrite the private/no-store directives set for per-user HTML and page-bids responses. Disable auction_html_comment debug flag in trusted-server.toml. * Restore nurl/burl/ad_id through adserver_mock mediation path The mock mediator endpoint does not echo nurl/burl/ad_id back in its response. Build a bid index in request_bids keyed by (provider, slot_id, bidder) — where bidder is recovered from the echoed crid field — and restore the fields in parse_mediation_response from the original SSP bids. Fixes the spec requirement: both nurl and burl must travel in __ts_bids for client-side sendBeacon firing on slotRenderEnded (§4.5). * Populate device.user_agent in auction request APS reads user agent from request.device — without it, real APS bids arrive with wrong or missing device targeting. Pass the incoming UA from both the page-load and page-bids auction paths. * Fix __tsAdInit fallback: look up bids by slot id not div id slotRenderEnded gives a div element id via getSlotElementId(), but __ts_bids is keyed by slot id. Build a divToSlotId map during slot setup (matching the TS implementation) and use it in the event handler. Without this, nurl/burl beacons and hb_adid match checks silently fail in the server-rendered fallback whenever div_id != id. * Format lint using cargo fmt * Fix clippy doc-markdown lint in adserver_mock * Remove inline PBS bidder params from creative-opportunities.toml PBS bidder credentials (mocktioneer, criteo placeholder params) were being sent directly to PBS on every auction request. Per the design spec, PBS bidder params belong in PBS stored requests keyed by slot ID, not in the edge config file. Removes PbsSlotParams struct, SlotProviders.pbs field, the to_ad_slot wiring block, and the corresponding test. Slots without inline bidder params trigger the existing storedrequest fallback path in the Prebid provider. Closes #697 * Clarify and test APS floor price enforcement in mediation path - Rewrite misleading comment in apply_floor_prices: price=None bids pass through in the parallel-only path because decoding is deferred; in the mediation path the mediator decodes prices before this function runs - Add test: decoded APS bid below slot floor is dropped - Add test: decoded APS bid at or above slot floor is kept Closes #698 * Document and test /auction API contract for non-Prebid.js callers - Expand handle_auction doc: inline-params vs stored-request paths, config passthrough and allowed_context_keys, response headers - Document AdRequest, AdUnit, BidConfig with the stored-request contract: absent/empty bids → empty bidders map → PBS stored-request fallback - Add tests for convert_tsjs_to_auction_request: - No bids → empty bidders map (stored-request path) - Inline bids → bidders map populated - Allowed config key passes through; disallowed key dropped - Invalid 3-element banner size returns error Closes #699 * Verify and document graceful degradation when no slots match URL - Add debug log at no-match gate in handle_publisher_request and handle_page_bids so operators can confirm the feature is inactive on non-article URLs without reading source code - Add test: empty slots file (kill-switch) returns slots:[] bids:{} - Add test: URL not matching any slot pattern returns slots:[] bids:{} Closes #700 * Format publisher.rs with cargo fmt * Document scroll/refresh handoff contract between TS and slim-Prebid - Clarify in handle_auction doc that /auction is for initial render and programmatic callers; scroll/refresh/SPA navigation is slim-Prebid's domain in Phase 1 - Note Phase 2 slot-template-aware refresh API as deferred future work - Add head_inserts doc clarifying __tsAdInit handles initial render only; slotRenderEnded fires win beacons but does not trigger refresh auctions Closes #702 * feat: add support for reading Extended User IDs from the ts-eids cookie in auction requests * feat: forward real client IP and geo to PBS in server-side auction Both handle_publisher_request and handle_page_bids now set device.ip and device.geo on the AuctionRequest after build_auction_request returns. Previously these were hardcoded to None, causing PBS to infer the IP from the Fastly edge IP — bidders like PubMatic filter such requests as non-human traffic. Client-side auction (formats.rs) already wired these fields. Server-side now matches that behaviour. * 20260518-053619 Review fixes for #680 (#708) * Tidy small review nits across the new auction surface Addresses review findings on #680: - P2-18 / price_bucket: reject NaN/Inf cpm up front before the (x * 100.0).floor() as u64 cast (Rust's NaN-cast behaviour is only safe by convention, not contract). Add test coverage. - P2-24 / publisher.rs::write_bids_to_state: drop the per-request log line from INFO to DEBUG so production logs don't spam a slot list on every page request. - P2-25 / publisher.rs::build_bids_script / build_ad_slots_script: serde_json::to_string of a Map / Vec is infallible -- use expect("should be infallible") instead of unwrap_or_else with a silent fallback that would mask any future bug. - P2-15 / prebid: drop the dead PrebidIntegrationConfig::suppress_nurl field -- declared but never read anywhere in the codebase. The no-op test it carried goes with it. * Extract GPT bootstrap script and guard double enableServices Addresses review findings on #680: - P2-2: the inline `__tsAdInit` bootstrap injected at <head> called googletag.pubads().enableSingleRequest() and googletag.enableServices() unconditionally. The TS bundle's later-installed version guards both with a `__tsServicesEnabled` flag — the inline version did not, so the publisher's own GPT init code (or an upgrade where the bundle loads before the bids script runs) caused double-enable + duplicate refresh(), producing duplicate ad requests on every load. Now both inline and bundle converge on the same flag and only invoke `refresh(newSlots)` for the slots this pass actually defined, never the global slot list. - P2-26: the bootstrap source moves out of a concat!() literal block in head_inserts() into a syntax-highlighted gpt_bootstrap.js file pulled in via include_str!. The Rust side keeps a single named constant, GPT_BOOTSTRAP_JS, so future edits diff cleanly. Adds a regression test that asserts the guard flag is present and that unbounded refresh() is gone. * Harden auction-orchestrator state cleanup Addresses review findings on #680: - P2-6: APS provider held its per-request slot_id_map across request boundaries when the same Wasm instance was reused (the mock provider already cleared its bid_index, APS did not). parse_response now `std::mem::take`s the map so it can never carry over to a subsequent request. - P2-7: apply_floor_prices used to silently pass bids with `price=None` through the floor filter. Today both production callers decode/skip None before calling, so the pass-through was dead code that would, if revived, cause winning_bids.len() to overcount what build_bid_map ships to the client. Drop the None branch and update the existing test to pin the new contract: callers must decode prices first. * Harden /__ts/page-bids and creative-opportunities loading Addresses review findings on #680: - P2-4: /__ts/page-bids ran the full SSP auction for every request without gating crawlers or prefetches the way the publisher path does, exposing partner request quota to client-side spraying. Apply the same is_bot / is_prefetch gate handle_publisher_request uses: slots are still returned (so HTML structure is unchanged) but the auction is short-circuited. New regression tests cover both gates. - P2-5 + P2-13: the adapter previously parsed CREATIVE_OPPORTUNITIES_TOML on every request and `expect("should parse...")` on failure, so a malformed embedded TOML (CI-bypassed binary patch, future schema change, anything build.rs didn't catch) would panic every request. Parse it lazily once per Wasm instance via LazyLock; on parse failure log an error and fall back to the documented "empty slots file = feature disabled" state instead of panicking. * Consolidate HTML stream processor + auction helpers Addresses review findings on #680: - P2-3 + P2-16: create_html_stream_processor previously built HtmlProcessorConfig inline, bypassing HtmlProcessorConfig::from_settings. A future edit to from_settings (e.g. to read a new flag from Settings) would silently miss the streaming-with-auction-hold path. Now goes through from_settings, with a new with_ad_state(...) builder method that layers the ad_slots_script / ad_bids_state fields on top. The `_settings` argument on create_html_stream_processor is now actually used and is no longer underscore-prefixed. - P2-17: the auction debug-comment prepend logic was duplicated between one_behind_loop (path=stream) and the BufferedProcessed arm (path=buffered). Extracted as `prepend_auction_debug_comment(label, result, state)` so the single source of truth gets one definition and the only difference between paths is the label string. - P2-14: build_auction_request was at the project's 7-argument cap. Bundled (matched_slots, request_path, co_config) into a new MatchedSlotsContext struct — the three fields always travel together anyway. The function now takes 5 args, leaving headroom for future per-request inputs without breaking the project rule. * Document AuctionContext request contract and pin mediator placeholder Addresses review finding P2-1 on #680: collect_dispatched_auction and the publisher-side collectors built fastly::Request::get("https://placeholder.invalid/") on the fly and stuffed it into AuctionContext.request before invoking the mediator. That worked today only because the current mediator (adserver_mock) does not read request headers. A future PBS-as-mediator change would silently lose DNT, client IP, UA, etc., because the placeholder has none of them. The structural fix that surfaces the contract: - AuctionContext::request now carries a thorough doc-comment that explicitly distinguishes the dispatch path (real client request, all headers available) from the collect path (synthetic placeholder, no client headers — mediators MUST snapshot at dispatch time if they need them). - MEDIATOR_PLACEHOLDER_URL is a single named const so the three inline `"https://placeholder.invalid/"` literals across orchestrator.rs and publisher.rs converge on one source of truth. Tests / debug-asserts can compare against it. - make_collect_context now debug_asserts that the placeholder argument matches the canonical URL, so any future caller that accidentally forwards a real client request through the collect path fails loudly in debug builds instead of triggering an invisible header-loss bug. A full snapshot-headers refactor (so mediators can read real client headers across the await) is tracked as follow-up; this PR makes the existing contract impossible to violate accidentally. * Apply cargo fmt to fix-up commits * Forward ts-eids cookie through /auction endpoint Addresses pass-2 review finding P3-1 on #680: handle_auction read the request's cookie jar for consent purposes but never threaded it into the AuctionRequest, so the Extended User IDs from the `ts-eids` cookie were dropped on the floor. The publisher-page and /__ts/page-bids paths both call parse_ts_eids_cookie(cookie_jar.as_ref()); the /auction endpoint now does the same so programmatic callers (slim-Prebid, native apps, server-to-server integrations) get parity instead of silently losing identity data. * Pass-3 review fixes for #680: bot helpers, dead arg, pre-compiled globs Addresses pass-2 review findings on #680: - Dedupe the bot-UA and prefetch checks. handle_publisher_request and handle_page_bids both inlined the same 5-entry crawler list and the sec-purpose/purpose header check. Promoted to two pub(crate) helpers in publisher.rs (is_bot_user_agent, is_prefetch_request) backed by a single BOT_USER_AGENT_FRAGMENTS const, so the two paths can't drift. - Drop the dead gam_network_id argument on CreativeOpportunitySlot::to_ad_slot. The parameter was already being discarded via `let _ = gam_network_id;`. Removing it also drops the now-unused co_config field from MatchedSlotsContext. - Pre-compile slot glob patterns once per Wasm instance. Each call to matches_path previously ran Pattern::new (per pattern, per slot, per request). The slots live in the adapter's LazyLock-cached CreativeOpportunitiesFile, so adding a #[serde(skip)] compiled_patterns cache populated by CreativeOpportunitiesFile::compile() at file-load time turns matches_path into a Vec<Pattern>::iter().any() lookup on the hot path. The fallback (compile-per-call) is preserved for hand-built slots in tests. Two new regression tests pin the cache. * Address deferred perf findings from PR #680 review P2-8: Promote STREAM_CHUNK_SIZE (8192) to module scope and use it for both the brotli Decompressor and CompressorWriter internal buffers, aligning them with the read buffer in one_behind_loop. P2-9: Replace RwLock<Option<String>> with Mutex<Option<String>> for ad_bids_state across publisher.rs and html_processor.rs. Single-threaded WASM gains no parallelism from RwLock; Mutex is simpler and avoids the reader-writer bookkeeping the workload cannot use. P2-10: Rewrite html_escape_for_script as a single-pass char loop with one pre-allocated String instead of seven sequential String::replace allocations. All existing escape tests pass unchanged. * Address PR review findings from #680 - Fix gpt_bootstrap.js APS beacon miss: listener now uses hb_bidder fallback when hb_adid is absent, matching the bundle's slotRenderEnded logic - Fix stale divToSlotId in inline listener: read from window.__tsDivToSlotId dynamically instead of local closure so SPA navigation updates are seen; early-return for slots not managed by Trusted Server - Populate window.__tsPrevGptSlots and window.__tsDivToSlotId from inline bootstrap so bundle's destroySlots and SPA nav path have correct state - Call installSlimPrebidLoader() in module init so the slim-Prebid lazy loader activates when __tsjs_slim_prebid_url is set; add three Vitest cases - Update /auction doc comment to distinguish /__ts/page-bids (SPA navigation) from /auction (initial render) and slim-Prebid (scroll/refresh) * Formatting * Address pass-4 review findings (#680) - Fix examnple.com typo in sourcepoint.rs test fixture - Guard SPA navigation race: check inflight controller identity after await res.json() before writing __ts_ad_slots/__ts_bids - Extract build_empty_bids_script() helper; html_processor.rs now calls it instead of duplicating the inline literal - Add invariant comment to unreachable None branch of prepend_auction_debug_comment - Cap parse_ts_eids_cookie to 32 eids / 32 uids per eid; log and return None when exceeded - Add #[serde(deny_unknown_fields)] to openrtb::Eid and Uid - Add #[serde(deny_unknown_fields)] to CreativeOpportunitiesFile and CreativeOpportunitySlot - Log debug message when adserver_mock crid does not match <bidder>-creative convention - Skip zero-dimension bids in adserver_mock with debug log - Fail closed in APS parse_aps_slot on malformed size string instead of producing 0x0 bid * Consolidate slot config into trusted-server.toml and namespace window globals under window._ts (#745) * Move slot templates from creative-opportunities.toml into trusted-server.toml Add [[creative_opportunities.slot]] array to trusted-server.toml and remove the separate creative-opportunities.toml file. Slots now deserialize directly into CreativeOpportunitiesConfig.slot via the existing vec_from_seq_or_map deserializer, with compile_slots() called in Settings::prepare_runtime(). Update publisher.rs and main.rs function signatures from &CreativeOpportunitiesFile to &[CreativeOpportunitySlot]. Build.rs slot-ID validation now reads from the merged settings rather than a separate file. * Namespace window globals under window.tsjs When slotRenderEnded fires before APS inserts its SafeFrame iframe (gamIframe is null), retry on the next animation frame. If the iframe appears by then, set its src to the adm creative URL. If still absent, replace slot content directly. Prevents the fixed_bottom sticky unit from being blanked when APS reveals it asynchronously after the event. * Wire KV-enriched EID resolution into server-side auction paths Both handle_publisher_request and handle_page_bids now run the full four-step EID pipeline (resolve_client_auction_eids → resolve_auction_eids → merge_auction_eids → gate_eids_by_consent) matching the client-side /auction endpoint. Previously both paths called parse_ts_eids_cookie, which read only the ts-eids browser cookie and skipped the KV identity graph lookup entirely. AuctionDispatch gains a registry field so the partner registry reaches handle_publisher_request without exceeding the seven-argument limit. handle_page_bids gains kv and registry parameters for the same reason. parse_ts_eids_cookie is moved to #[cfg(test)] as it is now test-only. * Remove dead build.rs from trusted-server-adapter-fastly The file only emitted a rerun-if-changed watch for creative-opportunities.toml, which was deleted when slot config was consolidated into trusted-server.toml. Config validation now runs entirely in trusted-server-core/build.rs. * Fix CI failures: update integration-tests lock file and prefer-const lint error * Update workspace Cargo.lock to resolve shared dependency version mismatch Aligns log (0.4.29 → 0.4.32) and serde_json (1.0.149 → 1.0.150) with the versions already pulled into crates/integration-tests/Cargo.lock. * Update spec to reflect consolidated slot config and current global namespace Replace all references to the deleted `creative-opportunities.toml` file with the `[creative_opportunities]` section in `trusted-server.toml`. Update all `window.__ts_*` global name references to the current `window.tsjs.*` namespace (tsjs.bids, tsjs.adSlots, tsjs.adInit). * Add suppress_nurl config and populate nurl/burl in mock test bids Add `suppress_nurl: bool` (default `false`) to `PrebidIntegrationConfig`. When `true`, strips `nurl` and `burl` from PBS bids at parse time, preventing client-side double-firing via `sendBeacon` when PBS fires win notifications server-side (ext.prebid.events.enabled). Update `adserver_mock` test-bidder response to include example `nurl`/`burl` values so the win-notification pipeline is exercisable in unit tests. * Fix cargo fmt * Fix server-side ad template streaming * Resolve server-side ad template review findings * Add per-bidder Prebid nurl suppression and refresh metadata * Resolve server-side ad template review issues * Resolve server-side ad template auction review findings - Fail closed on consent: add consent_allows_server_side_auction() helper requiring effective TCF/GPP Purpose 1 for GDPR and unknown jurisdictions (or any request carrying an EU TCF signal); used by both the publisher navigation auction and /__ts/page-bids - Pass the adapter's geo-aware EcContext into handle_page_bids so the jurisdiction decision sees real geo instead of always-unknown - Make [auction].enabled a real kill switch for the automatic publisher navigation ad stack and the /__ts/page-bids auction - Normalize Prebid server_url: use it as-is when it already ends with /openrtb2/auction, otherwise append the path (backward compatible) - Advertise the effective auction budget in provider payloads: PBS tmax and APS timeout now use the orchestrator-capped context.timeout_ms instead of raw provider config - Add a one-shot adInitRefreshInProgress bypass so slim-Prebid's refresh wrapper passes adInit()'s internal refresh straight to GPT instead of clearing server-side targeting with a duplicate client-side auction - Sweep stale TS targeting (hb_*, ts_initial, route keys) from all previously TS-touched GPT slots before applying a new SPA route - Map the GPT slot element ID (container div) in the inline bootstrap's divToSlotId so container-backed slots fire nurl/burl beacons * Resolve beacon, validation, and orchestrator review findings - Dedupe win/billing beacons: fire each bid's nurl/burl at most once, keyed by slot + bid identity in shared tsjs state so the inline bootstrap and bundle listeners can never double-fire; unify the ourBidWon check (hb_adid confirmation with hb_bidder fallback for APS bids) across both implementations - Wire validate_slot_id into Settings::prepare_runtime so every load path (including env-injected slots on runtime-config adapters) rejects invalid slot IDs; build.rs settings stub gains a no-op - Normalize the client-controlled page-bids path parameter: strip query/fragment and force a leading slash before glob matching - Document the deliberate Cache-Control private, max-age=0 choice (BFCache eligibility per design spec section 4.7, not no-store) - Align orchestrator collect path with the parallel path: use parse_response_with_context for providers and the mediator, and add a defense-in-depth deadline check to the collect select-loop - Migrate adserver_mock off request-scoped Mutex state: the SSP bid index is rebuilt in parse_response_with_context from the context's provider responses; document why APS's slot_id_map cannot follow yet - Make platform_response_to_fastly infallible; drop the dead error arms - Remove redundant PriceGranularity::dense and MediaType::banner constructors in favor of Default-based serde field defaults - Clarify that the Prebid stored-request fallback cannot fire for the client /auction path (every ad unit carries a trustedServer entry) - Consolidate GPT JS suites under test/integrations/gpt/, replace the leaked module-scope addEventListener patch with a restored wrapper, and add installSpaAuctionHook coverage (pushState/replaceState/ popstate, stale-response guard, non-OK response, idempotence) * Default tsjs adSlots and bids so gated-off pages never see undefined The edge injects adSlots and bids only when the server-side ad stack runs for the request. When it is gated off (kill switch, consent fail-closed, bots, prefetch), page code reading window.tsjs.bids or window.tsjs.adSlots previously threw on undefined and could blank SPA rendering. Core init now defaults them to {} / [] without clobbering edge-injected values that arrive before the bundle. Also flip the committed [auction] enabled default to true: with all provider integrations disabled and no slot templates in the checked-in config, the flag alone dispatches nothing, and the kill switch now actually gating the ad stack means a false default breaks every local/dev setup that enables a provider without noticing the flag. * Resolve failing integration tests * Remove unused test * Sync trusted-server.toml with main * Harden page-bids gate and keep surrogate headers off private responses Operator [response_headers] could reintroduce Surrogate-Control / Fastly-Surrogate-Control on responses the publisher path had marked private, making per-user HTML eligible for shared surrogate caching again. finalize_response now skips Cache-Control and both surrogate headers whenever the response already carries a private directive. GET /__ts/page-bids dispatched real PBS/APS auctions with no same-origin check, so any third-party page could trigger partner calls from a visitor's browser. The handler now requires Sec-Fetch-Site: same-origin/same-site, or the non-simple X-TSJS-Page-Bids header when fetch metadata is absent (legacy clients), and returns 403 otherwise. The tsjs SPA hook sends the header on every page-bids fetch. * Tighten page-bids origin gate and harden SPA bid application Address three review findings in the server-side ad-template flow: - page-bids gate now requires Sec-Fetch-Site: same-origin and no longer accepts same-site, which would admit sibling origins under the same registrable domain that are not trusted to spend SSP quota. The absent-metadata legacy fallback (X-TSJS-Page-Bids header) is unchanged. - The Prebid requestBids shim no longer drops inline server-side bidder params on a second call with the same ad unit. The first call prunes server-side bidders from unit.bids, so a later call rebuilt bidderParams as {}; it now retains the params captured on the first call when no new server-side bidders are present. - The SPA navigation hook defers applying bids until the new route's ad containers exist (MutationObserver, bounded by a 2s timeout) so a fast edge response cannot beat the DOM and silently drop server-side bids. * Scope ad-stack cache forcing and tighten win-beacon firing P1.2: handle_publisher_request forced Cache-Control: private and stripped surrogate headers on every text/html response, so a zero-slot or non-matching navigation lost shared cacheability even though no per-user ad data was injected. Gate that rewrite on should_run_ad_stack instead. First-visit identity responses are still protected: finalize_response now downgrades any Set-Cookie response to private and strips surrogate headers, since ec_finalize_response sets the EC cookie without a cache directive of its own. Returning-visitor, cookieless, non-ad HTML keeps its origin cache headers. P1.1: the slotRenderEnded win/billing beacon fired for any non-empty render whenever an APS bid existed for the slot (the !!hb_bidder fallback), over-reporting wins and billing for impressions won by other GAM demand. Require an hb_adid match instead, in both the bundle listener and the inline bootstrap. Slot targeting is still request state rather than proof of the winning line item, but this removes the unconditional false positive. * Add Prebid Cache fields to make_bid test helper after main merge Merging main brought back the formats.rs make_bid helper, whose Bid literal predated the cache_id/cache_host/cache_path/ad_id fields this branch added to auction::types::Bid. Set them to None so the lib test target compiles. Pure test-helper fix; no change to /auction behavior. * Fix server-side ad runtime review issues * Bind render bridge to source slot and fix refresh parity Resolve three #680 review findings on the server-side ad runtime: - Render bridge now requires the requesting iframe's slot to own the resolved hb_adid before responding or firing win/billing beacons. Previously an iframe under slot A could request slot B's adId and receive slot B's creative while firing slot B's beacons. - Refresh ad units now include configured client-side bidders by merging matching pbjs.adUnits bid entries, so native Prebid demand is not dropped on refresh/scroll impressions. - Inline GPT bootstrap wraps its internal refresh with the adInitRefreshInProgress sentinel, mirroring the TS adInit so a pre-installed slim-Prebid refresh wrapper does not clear TS targeting. Add regression tests for the two-slot render-bridge mismatch and the client-side bidder refresh merge. * Address server-side ad review comments * Restore publisher platform-http-client test on the merged signature Re-add publisher_request_uses_platform_http_client_with_http_types, dropped during the main merge because it called the pre-feature 4-arg handle_publisher_request. A run_publisher_proxy test helper supplies the no-auction EC/AuctionDispatch wiring so the test body stays a plain (settings, registry, services, req) proxy call. * Gate POST /auction behind the server-side auction consent check The publisher-navigation and /__ts/page-bids paths fail closed for GDPR or unknown jurisdictions that lack effective TCF Purpose 1, but POST /auction proceeded straight to run_auction after only stripping EC IDs/EIDs — still dispatching PBS/APS calls and forwarding request-derived signals (UA/IP/geo, and cookies under some Prebid consent-forwarding modes) for traffic the gate says must not run a server-side auction. Apply consent_allows_server_side_auction before resolving EIDs or contacting providers; when it denies, return an empty no-bid OpenRTB response without invoking run_auction. Add a regression test that registers a panic-on-bid provider and proves a GDPR/unknown request lacking Purpose 1 returns no bids without contacting any provider. Route the orchestration-failure /auction tests through a non-regulated geo so they still exercise the provider path. * Validate creative-opportunity slots at build time build.rs deserialized slots into a stub whose validate_runtime was a no-op and only checked slot-id syntax, so an invalid trusted-server.toml or TRUSTED_SERVER__CREATIVE_OPPORTUNITIES__SLOT override (empty page_patterns, empty formats, zero dimensions, empty resolved GAM unit path) passed CI and got embedded, then failed at request time as a configuration error. Extract the validation into creative_slot_build_check, shared by build.rs (via #[path]) and the crate test build (via #[cfg(test)] mod) so the rules run under cargo test. It mirrors CreativeOpportunitySlot::validate_runtime and runs against the merged config (base TOML plus TRUSTED_SERVER__* env overrides) before the config is serialized and embedded, so an invalid slot fails the build and is never persisted. * Restore mediated render/accounting fields on the synchronous auction path run_parallel_mediation parsed the mediator response through parse_response, which (for adserver_mock) drops nurl/burl/ad_id and PBS cache fields restored only in parse_response_with_context. The synchronous mediation path used by POST /auction and /__ts/page-bids could therefore return mediated cache bids without hb_adid / cache metadata, breaking creative rendering and win/billing beacons even though the dispatched collect path preserves them. Call parse_response_with_context with the mediator context (which carries the collected SSP responses), matching the dispatched collect path. Add a regression test proving a mediated bid keeps its restored nurl/ad_id through run_auction. * Display TS-defined GPT slots instead of only refreshing them In the fallback path where Trusted Server defines a GPT slot itself, the code called defineSlot().addService() then refresh(), but never googletag.display() for the new slot. GPT requires a display() call to register/render a slot, so TS-owned first-impression slots no-op ("defineSlot was called without a matching display call") and miss impressions. Reused publisher-owned slots are unaffected because the publisher already displayed them. Track TS-defined slot element IDs separately, display() them once after services are enabled, and keep refresh() for reused publisher-owned slots only. Mirror the change in the inline gpt_bootstrap.js. Add Vitest coverage for the TS-owned display path and keep the refresh-bypass test on a reused slot. * Validate creative-opportunity glob patterns at build time The build-time validator only checked for a non-empty page_patterns string, so a config like page_patterns = ["["] passed the release build and then failed settings load at runtime when compile_patterns rejected the slot. Compile each pattern with the same glob::Pattern::new + ** -> * normalization contract used by the runtime compile_patterns, requiring at least one pattern that compiles. Adds glob as a build-dependency and tests for an uncompilable pattern and the recursive ** case. * Match Cache-Control privacy directives case-insensitively finalize_response checked for lowercase "private"/"no-store" substrings, but Cache-Control directives are case-insensitive (RFC 9111). A Cache-Control: No-Store on a Set-Cookie response was treated as cacheable and downgraded to the weaker private, max-age=0, and a Cache-Control: Private did not block operator response_headers from re-enabling shared caching. Lowercase the header value before matching. Add mixed-case No-Store / Private tests. * Align checked-in creative auction timeout with its 500ms guidance The comment recommends a 500ms default because the value bounds the DOMContentLoaded/window.load slip, but the checked-in value was 1500ms, so a first rollout that enables slots while inheriting the default would impose a 1.5s close-body hold on cache-hit pages. Set the sample default to 500ms. * Correct float-truncation under-bucketing in price_bucket Many two-decimal CPMs are not exactly representable in binary floating point: 0.29 * 100.0 is 28.999…, so flooring truncated it to 28 ("0.28"), and 1.15 became "1.14". These values feed hb_pb targeting keys, so the auction reported a cent low. Convert CPM to whole cents through a helper that nudges values sitting an ULP below a cent boundary up before flooring, leaving genuinely sub-cent values (0.015 -> "0.01") untouched. Adds a float-boundary regression test. * Cap synchronous mediator timeout to its configured budget run_parallel_mediation gave the mediator the full remaining auction budget, while the dispatched collect path bounds it by remaining.min(mediator.timeout_ms()). Apply the same cap for symmetry between the two paths. * Warn when a dispatched auction is dropped on non-streaming routes should_run_auction is decided from request signals before the origin content-type/status/encoding is known. A navigation that dispatched SSP bid requests but then routes to PassThrough (2xx non-HTML) or BufferedUnmodified (non-2xx, unsupported encoding, empty host) dropped the DispatchedAuction without collecting it — wasted SSP quota with no visibility. Log a warning on those arms when an auction was dispatched. * Test env-injected creative-opportunity slot-id rejection Lock in that a TRUSTED_SERVER__CREATIVE_OPPORTUNITIES__SLOT override with an invalid id is rejected through from_toml_and_env, complementing the existing TOML-path test and exercising the same validation the build-time check uses. * Use a valid glob as the page-pattern doc example "/20**" is an invalid glob that only matches via the **->* normalization fallback; using it as the canonical example invites copy-paste of broken config. Show "/2024/*" as the primary example and keep the normalization note as the edge-case caveat. * Remove dead test-only parse_ts_eids_cookie helper parse_ts_eids_cookie was gated to #[cfg(test)] and exercised only by its own tests; production reads the ts-eids cookie through resolve_client_auction_eids -> parse_prebid_eids_cookie (which enforces its own size/length caps). Remove the function, its tests, and the now-orphaned cfg(test) imports/helpers. * Force EC Set-Cookie responses to stay shared-uncacheable finalize_response applies the cookie cache-privacy downgrade on the HttpResponse, but the EC identity cookie is written later by ec_finalize_response onto the converted Fastly response. A first-visit navigation whose only per-user payload is the EC cookie therefore kept any public/surrogate cache headers from the origin or operator response headers, so a shared cache could store and replay one visitor's EC cookie to others. Re-apply the downgrade with enforce_set_cookie_cache_privacy after EC finalization in both the buffered and streaming branches, mirror it in the route test helper, and cover the first-visit ordering with route tests. * Preserve server-side bidder params on Prebid refresh auctions The synthetic refresh ad unit only carried the trustedServer bid with a zone, so the requestBids shim had no original server-side bidder entries to collect into bidderParams. Refresh/scroll /auction requests therefore sent {} for inline PBS params and dropped demand the publisher configured only on the initial ad unit. Recover the matching original pbjs.adUnits server-side params by ad unit code — from both raw bidder entries and params already folded onto the initial trustedServer bid — and attach them to the synthetic refresh bid. * Reject build-time creative-opportunity configs the runtime can't load The build script types price_granularity as a String and slots as raw JSON values, so values the runtime schema rejects — a price_granularity outside the PriceGranularity enum (e.g. custom), or unknown slot keys under the slot's deny_unknown_fields — embedded cleanly and then failed settings load on every non-health request, turning a green build into a request-time outage. Validate price_granularity against the real PriceGranularity enum and reject unknown top-level slot fields in the shared build-check validator before the merged config is embedded, with tests for both. * Close EC and ad-stack gaps in page-bids and publisher flow Stop handle_publisher_request from minting its own EC ID. EC generation is the adapter's real-browser-gated responsibility; the duplicate inline call re-ran for any navigation with no real-browser signal, so a non-real-browser client could get an IP-derived EC minted in memory and forwarded to PBS/APS even though the adapter blocked EC operations. Gate /__ts/page-bids slot output on the effective ad-stack condition (auction kill switch + consent), not just winning bids. Returning slots while the stack is disabled let the SPA hook run adInit() and create or refresh GPT slots client-side, defeating the kill switch. This matches the publisher navigation path's should_run_server_side_ad_stack gate. Add deny_unknown_fields to the top-level creative-opportunities config and nested provider/format structs so misspelled keys fail at startup instead of silently disabling or mis-timing the ad stack. Add regression tests for all three and update the page-bids tests to isolate the bot/prefetch variable from the consent gate. * Close cache-privacy and refresh-recovery gaps from PR review Apply request-filter response effects before the final Set-Cookie cache guard in every Fastly response path (buffered, streaming, asset streaming) so a per-user cookie added by a DataDome allow can no longer leave with public/surrogate cache headers. Strip surrogate cache headers on every Set-Cookie response — even one keeping a stricter no-store directive — and treat no-store as protected in the operator-header guard. Reject OPTIONS /__ts/page-bids at the adapter so the side-effecting endpoint never grants a CORS preflight the publisher origin might. Drain every dispatched SSP request in the collect loop instead of breaking on the auction deadline, so a slow origin can no longer discard SSP responses that already arrived. Reject empty/whitespace div_id overrides at runtime validation, which would otherwise bind a slot to the first id-bearing DOM element. Recover Prebid refresh params and client-side bids from candidate codes ([gpt element id, injected div_id]) so container-backed slots keep the publisher's configured demand on refresh/scroll auctions. * Close EID-consent and GPT initial-load gaps from PR review Gate /auction client EID resolution on the same identity-consent condition as the EC ID (`ec_id.is_some()`, already filtered by `ec_allowed()`). Previously client-provided EIDs from the request body or ts-eids cookie were resolved unconditionally, so a US/GPC or US-Privacy opt-out context — where EC identity use is denied but a non-personalized auction may still run — could forward persistent EIDs, since `gate_eids_by_consent` only strips on TCF/GDPR signals. This matches the publisher and /__ts/page-bids paths. Refresh TS-defined GPT slots when the publisher disabled initial load. With pubads().disableInitialLoad(), display() only registers a freshly defined slot and the ad request must come from refresh(); TS-owned first-impression slots were only display()ed, so they rendered blank. A wrapper around disableInitialLoad() records the state on window.tsjs, and adInit() refreshes its own slots when it is set (bundle and gpt_bootstrap.js). The detector only hooks an existing googletag stub so a plain import never touches window.googletag. * Close build/runtime validation parity and observability gaps from PR review Address PR #680 review findings: Blocking build/runtime parity: - Remove the dead glob stub in build.rs so creative-slot page-pattern validation runs against the real glob crate. An invalid pattern such as `["["]` now fails the build instead of being embedded and dropped at runtime settings load. - Reject an empty/whitespace div_id override at build time, mirroring CreativeOpportunitySlot::validate_runtime. - Validate nested creative-slot fields (formats, providers, aps, prebid) against the runtime structs' deny_unknown_fields so env-injected typos like `mediatype` or `slotId` fail the build, not runtime. Observability and correctness: - Mirror the parallel auction path on the dispatch/collect path: attribute provider parse failures (error_type + message) and transport failures (via failed_backend_name) in provider_details. - Warn on each page pattern dropped during compile_patterns so a mixed valid/invalid set is visible to operators. - Escape the </script> terminator in the configured slim_prebid_url so it cannot break out of its inline script tag. - Guard SPA navigation: onNavigate no-ops when the path is unchanged, so popstate (hash-only or same-path back/forward) no longer re-requests impressions. Docs and comments: - Update the GPT scroll/refresh handoff comment to reflect installSpaAuctionHook + /__ts/page-bids ownership of SPA navigation. - Note that targeting.zone is not forwarded when explicit prebid.bidders are set. - Split the page-bids same-origin-gate and path-normalization docs onto their own functions; remove the stale # Panics section on handle_publisher_request. - Correct the stale slotRenderEnded/beacon comment in gpt_bootstrap.js. Tests added for div_id, nested-field, slim_prebid_url escaping, and SPA same-path guard behavior. * Ignore leftover artifacts in pre-rename crate dirs The EdgeZero sync (#761) renamed crates/js and crates/integration-tests to crates/trusted-server-*. The old directories still hold local-only build artifacts (node_modules, target, dist) whose gitignore rules moved with the rename, so git now sees them as untracked. Ignore the defunct paths until the directories are removed from disk. * Address PR #680 review findings P1 — EdgeZero finalize cache/Set-Cookie privacy parity: Share the protected finalizer between the legacy and EdgeZero paths. apply_finalize_headers now strips surrogate cache headers and downgrades cookie-bearing responses to private, and skips operator response_headers that would re-enable shared caching on uncacheable responses; finalize_response delegates to it. The EdgeZero entry point re-applies an HttpResponse enforce_set_cookie_cache_privacy after ec_finalize_response and request-filter effects so a late EC Set-Cookie cannot reach a shared cache. Adds middleware tests for both cases. P1 — empty page-bids must not enable GPT services: adInit() only enables GPT services when it has a slot to display or refresh, and the SPA hook skips adInit() for an empty page-bids response unless prior TS state needs sweeping. Prevents a consent-denied or kill-switched navigation from activating the publisher's GPT setup. P2 — scope Prebid refresh targeting to the refreshed slots: setTargetingForGPTAsync is called with the synthetic refresh ad-unit codes so a one-slot refresh no longer mutates unrelated GPT slots. P2 — validate nested slot value shapes at build time: The creative-slot build check now validates media_type against the runtime MediaType variants, targeting as a string map, page_patterns as strings, providers.aps.slot_id as a string, providers.prebid.bidders as a map, and floor_price as a number — closing build-green/runtime-broken gaps. A drift-guard test ties media_type to the runtime enum. CI — suppress CodeQL cleartext-logging false positives: Annotate the provider/mediator "not registered" warnings; they log static config identifiers, not secrets. * Add glob to the integration-tests lockfile The merge took main's trusted-server-integration-tests Cargo.lock, but the branch's trusted-server-core now pulls in glob (the creative-slot build check uses glob::Pattern). The integration crate path-depends on core, so its locked graph was missing glob and the --locked CI build refused to update it. Add only glob v0.3.3; no other versions change, keeping the shared direct-dependency parity check green. * Fix server-side ad template review blockers * Fix EdgeZero empty ad-template config gate * Address fourth-pass PR review findings Blocking: - Move tokio to [dev-dependencies] in trusted-server-core; it was only used by #[tokio::test] and was linking the runtime into the wasm prod build. Confirmed the release wasm adapter build no longer pulls tokio. - Roll back the SPA currentPath on a failed /__ts/page-bids fetch so a transient error no longer permanently strands that route (gpt/index.ts). Build/runtime parity and diagnostics: - Bound creative-opportunity format width/height to u32 range at build time so values the runtime u32 cannot hold are rejected early. - Add #[serde(deny_unknown_fields)] to the build.rs config stub to match the runtime type and reject mistyped table keys at build time. - Warn when the <body> end-tag handler is absent so a silently non-rendering server-side ad feature is diagnosable. - Log dropped slot bidders that are neither configured nor the aps provider. - Log build_bid_index collisions (multiple bids per seat/imp). JS correctness: - Narrow uid.atype to a number before the range check in sanitizeAuctionUid. - Resolve findInjectedSlotForRefresh by exact/container match before the prefix fallback, with a regression test for prefix-overlapping div_ids. - Guard the gpt_bootstrap prefix scan against an empty div_id. - Route injectAdmIntoSlot through findSlotElementByDivId for consistency. Cleanup and docs: - Remove the dead has_post_processors routing dependency from classify_response_route and (now unused) handle_publisher_request. - Extract the duplicated EID resolution/consent-gating/device tail shared by the initial-page and page-bids dispatch paths into one helper. - Anchor the surrogate cache-header list in a shared const so the legacy and EdgeZero Set-Cookie privacy paths stay aligned. - Refresh stale docs (PublisherResponse::Stream, the publisher module platform-coupling note, and UserInfo.eids consent-gate location). * Drop tokio from the integration-tests lockfile Moving tokio to trusted-server-core dev-dependencies removed it from the crate's normal dependency list, so the integration-tests lockfile (which resolves core's non-dev deps) no longer pins tokio under core. Keeps `cargo --locked` green for the integration job. * Run the server-side auction on the Axum, Cloudflare, and Spin adapters These EdgeZero-style adapters finalize buffered, and the sync `buffer_publisher_response` drives `stream_publisher_body`, which ignores `params.dispatched_auction` — so they injected an empty `tsjs.bids = {}` while Fastly (legacy streaming finalize) served real bids. - Add `buffer_publisher_response_async` in core: for the Stream variant it drives `stream_publisher_body_async`, which awaits `collect_dispatched_auction`, writes `ad_bids_state`, and injects the bids before `</body>`. - Pass the configured `creative_opportunities.slot` (not empty) to `handle_publisher_request` on all three adapters; it matches them against the request path internally. - Call the async finalize from each adapter (Cloudflare/Spin via their now-async `resolve_publisher_response`). EID targeting stays off for now (these adapters pass `kv: None`). * Build the EC consent context from the request on the portability adapters The auction consent gate (`consent_allows_server_side_auction`)…
1 parent 0084429 commit c058a57

67 files changed

Lines changed: 17923 additions & 1537 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,15 @@ TRUSTED_SERVER__REQUEST_SIGNING__ENABLED=false
3737

3838
# Prebid
3939
TRUSTED_SERVER__INTEGRATIONS__PREBID__ENABLED=false
40-
# TRUSTED_SERVER__INTEGRATIONS__PREBID__SERVER_URL=https://prebid-server.com/openrtb2/auction
40+
# TRUSTED_SERVER__INTEGRATIONS__PREBID__SERVER_URL=https://prebid-server.example.com/openrtb2/auction
4141
# TRUSTED_SERVER__INTEGRATIONS__PREBID__TIMEOUT_MS=1000
4242
# TRUSTED_SERVER__INTEGRATIONS__PREBID__BIDDERS=kargo,rubicon,appnexus
4343
# TRUSTED_SERVER__INTEGRATIONS__PREBID__BID_PARAM_OVERRIDES='{"bidder-name":{"param1":12345,"param2":"value"}}'
4444
# Compatibility env shape for bidder -> zone -> params overrides
4545
# TRUSTED_SERVER__INTEGRATIONS__PREBID__BID_PARAM_ZONE_OVERRIDES='{"kargo":{"header":{"placementId":"_abc"}}}'
4646
# Preferred canonical env shape for future generic rules
4747
# TRUSTED_SERVER__INTEGRATIONS__PREBID__BID_PARAM_OVERRIDE_RULES='[{"when":{"bidder":"kargo","zone":"header"},"set":{"placementId":"_abc"}}]'
48+
# TRUSTED_SERVER__INTEGRATIONS__PREBID__SUPPRESS_NURL_BIDDERS=exampleBidder,anotherBidder
4849
# TRUSTED_SERVER__INTEGRATIONS__PREBID__AUTO_CONFIGURE=false
4950
# TRUSTED_SERVER__INTEGRATIONS__PREBID__DEBUG=false
5051
# TRUSTED_SERVER__INTEGRATIONS__PREBID__TEST_MODE=false

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,9 @@ src/*.html
5353
/crates/trusted-server-integration-tests/browser/test-results/
5454
/crates/trusted-server-integration-tests/browser/playwright-report/
5555
/crates/trusted-server-integration-tests/browser/.browser-test-state.json
56+
57+
# Defunct pre-rename crate dirs (renamed to crates/trusted-server-*); ignore the
58+
# leftover local build artifacts (node_modules, target, dist) that remain on disk.
59+
/crates/js/
60+
/crates/integration-tests/
61+

Cargo.lock

Lines changed: 8 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ fastly = "0.12"
5555
fern = "0.7.1"
5656
flate2 = "1.1"
5757
futures = "0.3"
58+
glob = "0.3"
5859
getrandom = "0.2"
5960
hex = "0.4.3"
6061
hmac = "0.12.1"

crates/trusted-server-adapter-axum/src/app.rs

Lines changed: 95 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ use trusted_server_core::proxy::{
1919
handle_first_party_proxy_sign,
2020
};
2121
use trusted_server_core::publisher::{
22-
buffer_publisher_response, handle_publisher_request, handle_tsjs_dynamic,
22+
AuctionDispatch, buffer_publisher_response_async, handle_page_bids, handle_publisher_request,
23+
handle_tsjs_dynamic, page_bids_preflight_denied,
2324
};
2425
use trusted_server_core::request_signing::{
2526
handle_trusted_server_discovery, handle_verify_signature,
@@ -134,6 +135,34 @@ where
134135
.unwrap_or_else(|e| http_error(&e)))
135136
}
136137

138+
// ---------------------------------------------------------------------------
139+
// EC context
140+
// ---------------------------------------------------------------------------
141+
142+
/// Builds the geo-aware [`EcContext`] for consent-gated endpoints (`/auction`,
143+
/// `/__ts/page-bids`, and the publisher fallback).
144+
///
145+
/// Mirrors the Fastly entry point: `EcContext::default()` leaves jurisdiction
146+
/// Unknown, which fails the auction consent gate closed even for consented
147+
/// users. Geo comes from the platform (a no-op on the local Axum dev server, so
148+
/// jurisdiction stays Unknown there unless the request carries TCF consent). A
149+
/// malformed consent string is logged and falls back to the default
150+
/// (fail-closed) context rather than being silently swallowed.
151+
fn build_ec_context(state: &AppState, services: &RuntimeServices, req: &Request) -> EcContext {
152+
let geo_info = services
153+
.geo()
154+
.lookup(services.client_info().client_ip)
155+
.unwrap_or_else(|e| {
156+
log::warn!("geo lookup failed: {e}");
157+
None
158+
});
159+
EcContext::read_from_request_with_geo(&state.settings, req, services, geo_info.as_ref())
160+
.unwrap_or_else(|e| {
161+
log::warn!("EC context read failed: {e:?}");
162+
EcContext::default()
163+
})
164+
}
165+
137166
// ---------------------------------------------------------------------------
138167
// Fallback dispatcher (tsjs / integration proxy / publisher)
139168
// ---------------------------------------------------------------------------
@@ -171,9 +200,34 @@ async fn dispatch_fallback(
171200
});
172201
}
173202

174-
handle_publisher_request(&state.settings, &state.registry, services, req)
175-
.await
176-
.and_then(|pr| buffer_publisher_response(pr, &method, &state.settings, &state.registry))
203+
// Run the server-side auction with the configured creative-opportunity
204+
// slots; `handle_publisher_request` matches them against the request path.
205+
let mut ec_context = build_ec_context(state, services, &req);
206+
let auction = AuctionDispatch {
207+
orchestrator: &state.orchestrator,
208+
slots: state.settings.creative_opportunity_slots(),
209+
registry: None,
210+
};
211+
let publisher_response = handle_publisher_request(
212+
&state.settings,
213+
services,
214+
None,
215+
&mut ec_context,
216+
auction,
217+
req,
218+
)
219+
.await?;
220+
// Async finalize so the dispatched auction is collected and its bids are
221+
// injected before `</body>` (the sync buffer path would drop them).
222+
buffer_publisher_response_async(
223+
publisher_response,
224+
&method,
225+
&state.settings,
226+
&state.registry,
227+
&state.orchestrator,
228+
services,
229+
)
230+
.await
177231
}
178232

179233
fn fallback_handler(
@@ -202,6 +256,7 @@ enum NamedRouteHandler {
202256
/// reach the publisher fallback (which would leak admin credentials).
203257
LegacyAdminDenied,
204258
Auction,
259+
PageBids,
205260
FirstPartyProxy,
206261
FirstPartyClick,
207262
FirstPartySign,
@@ -224,7 +279,7 @@ const LEGACY_ADMIN_DENY_METHODS: &[Method] = &[
224279
Method::DELETE,
225280
];
226281

227-
fn named_routes() -> [NamedRoute; 11] {
282+
fn named_routes() -> [NamedRoute; 12] {
228283
[
229284
NamedRoute {
230285
path: "/.well-known/trusted-server.json",
@@ -270,6 +325,13 @@ fn named_routes() -> [NamedRoute; 11] {
270325
primary_methods: &[Method::POST],
271326
handler: NamedRouteHandler::Auction,
272327
},
328+
// GET runs the SPA re-auction; OPTIONS is denied in-handler as a CORS
329+
// preflight guard for this side-effecting endpoint.
330+
NamedRoute {
331+
path: "/__ts/page-bids",
332+
primary_methods: &[Method::GET, Method::OPTIONS],
333+
handler: NamedRouteHandler::PageBids,
334+
},
273335
NamedRoute {
274336
path: "/first-party/proxy",
275337
primary_methods: &[Method::GET],
@@ -328,7 +390,10 @@ fn named_route_handler(
328390
}
329391
NamedRouteHandler::LegacyAdminDenied => Ok(legacy_admin_alias_denied()),
330392
NamedRouteHandler::Auction => {
331-
let ec_context = EcContext::default();
393+
// Build the geo-aware EC context so the auction consent
394+
// gate sees the caller's jurisdiction — `EcContext::default()`
395+
// fails it closed for consented users.
396+
let ec_context = build_ec_context(&state, &services, &req);
332397
handle_auction(
333398
&state.settings,
334399
&state.orchestrator,
@@ -340,6 +405,30 @@ fn named_route_handler(
340405
)
341406
.await
342407
}
408+
NamedRouteHandler::PageBids => {
409+
// SPA re-auction endpoint. `OPTIONS` is a CORS preflight
410+
// for this side-effecting GET and is always denied so the
411+
// GET handler's `X-TSJS-Page-Bids` gate stays trustworthy.
412+
if req.method() == Method::OPTIONS {
413+
Ok(page_bids_preflight_denied())
414+
} else {
415+
let ec_context = build_ec_context(&state, &services, &req);
416+
let auction = AuctionDispatch {
417+
orchestrator: &state.orchestrator,
418+
slots: state.settings.creative_opportunity_slots(),
419+
registry: None,
420+
};
421+
handle_page_bids(
422+
&state.settings,
423+
&services,
424+
None,
425+
auction,
426+
&ec_context,
427+
req,
428+
)
429+
.await
430+
}
431+
}
343432
NamedRouteHandler::FirstPartyProxy => {
344433
handle_first_party_proxy(&state.settings, &services, req).await
345434
}

crates/trusted-server-adapter-axum/src/middleware.rs

Lines changed: 8 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ use std::sync::Arc;
33
use async_trait::async_trait;
44
use edgezero_core::context::RequestContext;
55
use edgezero_core::error::EdgeError;
6-
use edgezero_core::http::{HeaderName, HeaderValue, Response};
6+
use edgezero_core::http::{HeaderValue, Response};
77
use edgezero_core::middleware::{Middleware, Next};
88
use trusted_server_core::auth::enforce_basic_auth;
99
use trusted_server_core::constants::HEADER_X_GEO_INFO_AVAILABLE;
@@ -88,26 +88,19 @@ impl Middleware for AuthMiddleware {
8888
///
8989
/// Unlike the Fastly variant, geo is always unavailable so `X-Geo-Info-Available: false`
9090
/// is unconditionally emitted. Fastly-specific headers are omitted.
91-
/// Operator-configured `settings.response_headers` are applied last and can override
92-
/// any managed header.
91+
/// Operator-configured `settings.response_headers` are applied last (with the
92+
/// shared cookie cache-privacy hardening) and can override any managed header.
9393
pub(crate) fn apply_finalize_headers(settings: &Settings, response: &mut Response) {
9494
response.headers_mut().insert(
9595
HEADER_X_GEO_INFO_AVAILABLE,
9696
HeaderValue::from_static("false"),
9797
);
9898

99-
for (key, value) in &settings.response_headers {
100-
let header_name = HeaderName::from_bytes(key.as_bytes());
101-
let header_value = HeaderValue::from_str(value);
102-
if let (Ok(header_name), Ok(header_value)) = (header_name, header_value) {
103-
response.headers_mut().insert(header_name, header_value);
104-
} else {
105-
log::warn!(
106-
"Skipping invalid configured response header value for {}",
107-
key
108-
);
109-
}
110-
}
99+
// Cookie-bearing responses stay private to shared caches and operator
100+
// headers cannot re-enable caching for uncacheable per-user payloads.
101+
trusted_server_core::response_privacy::apply_response_headers_with_cache_privacy(
102+
settings, response,
103+
);
111104
}
112105

113106
// ---------------------------------------------------------------------------

crates/trusted-server-adapter-axum/src/platform.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -642,6 +642,7 @@ mod tests {
642642
port: None,
643643
certificate_check: true,
644644
first_byte_timeout: Duration::from_secs(15),
645+
between_bytes_timeout: Duration::from_secs(15),
645646
host_header_override: None,
646647
};
647648
let name1 = backend.predict_name(&spec).expect("should return a name");
@@ -661,6 +662,7 @@ mod tests {
661662
port: None,
662663
certificate_check: true,
663664
first_byte_timeout: Duration::from_secs(15),
665+
between_bytes_timeout: Duration::from_secs(15),
664666
host_header_override: None,
665667
};
666668
assert_eq!(

0 commit comments

Comments
 (0)