Commit 53131fb
feat(sources): rss provider phase 3/4 — freshness engine, partition-per-feed exec, table providers, catalog registration
Query surface of the RSS/Atom provider (plan Tasks 11-14):
- RssEngine: the per-feed freshness state machine (TTL hit, conditional
revalidation, 304 re-arm, refetch, degrade-to-stale) — sole production
consumer of egress/fetch/cache, whose dead_code allowances drop here
- RssScanExec: one partition per feed, concurrency bounded by
max_concurrent, LIMIT gates further fetch launches after acquire, scan
deadline degrades a partition to zero rows; reset_state overridden so
re-execution (RecursiveQueryExec) starts clean
- feeds/items TableProviders: exact pushdown for single-feed equality,
IN lists, and single-column equality disjunctions (what DataFusion's
simplifier rewrites short IN lists into); pruning matches pushdown
- register_rss_tables: one source = one catalog (<name>.main.feeds /
<name>.main.items), zero network I/O at registration; the egress
policy is constructed in exactly one production place
Part 3 of 4 stacked PRs replacing #175 (base: rss-m1-phase2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>1 parent cee644d commit 53131fb
7 files changed
Lines changed: 4743 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | 68 | | |
77 | 69 | | |
78 | 70 | | |
| |||
Large diffs are not rendered by default.
0 commit comments