Skip to content

Commit 34515ef

Browse files
gracexmatinclaude
andcommitted
feat(sources): rss provider phase 4/4 — server wiring, acceptance suites, fixture corpus, M2 docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent f656780 commit 34515ef

41 files changed

Lines changed: 5471 additions & 1 deletion

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,7 @@ For end-to-end walkthroughs — RAG, recommendations, an agent-native wiki, a si
338338
| InfluxDB 3 | Read | No | Time-series measurements over Arrow Flight SQL | [docs/influxdb/](docs/influxdb/) |
339339
| Open Connector | Read | Yes | SaaS resources as stable SQL tables via a self-hosted [Open Connector](https://github.com/oomol-lab/open-connector) gateway; GitHub pack (repos, issues, PRs, reviews, commits, workflow runs, releases — [guide](docs/open-connector-github.md)), Slack pack (conversations, users, files — [guide](docs/open-connector-slack.md)), `open_connector_query` / `open_connector_scan` UDTFs, filter + limit pushdown, bounded TTL cache (more provider packs rolling out) | [docs/open-connector.md](docs/open-connector.md), [demo](docs/open-connector/) |
340340
| Documents | Read | No | PDF/Office/ODF/image -> per-page markdown, tables, images (local directories or S3 prefixes; `documents` feature) | [docs/documents.md](docs/documents.md) |
341+
| RSS / Atom | Read | Yes | RSS 0.9x/1.0/2.0, Atom, JSON Feed subscriptions as `feeds` (health) + `items` (live window); content stored as Markdown, per-feed TTL cache with conditional GETs, per-feed fault isolation, un-sandboxed fetch egress — operator/Cloud-owned SSRF control (`rss` feature) | [docs/rss.md](docs/rss.md) |
341342

342343
---
343344

@@ -363,6 +364,8 @@ For end-to-end walkthroughs — RAG, recommendations, an agent-native wiki, a si
363364

364365
</details>
365366

367+
Most sources read a store that already holds rows. The **RSS/Atom** source is the one that reaches out over the open web at query time: it fetches each subscription through a per-feed TTL cache with conditional GETs, runs one DataFusion partition per feed so a dead feed degrades alone instead of failing the scan, and — because feed URLs are agent-authored input — the fetch is deliberately un-sandboxed in OSS: Skardi does not restrict where a feed may point, and operators impose egress control at the infrastructure layer (managed egress is a Skardi Cloud feature). See [docs/rss.md](docs/rss.md).
368+
366369
---
367370

368371
## Docker

crates/server/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ chunking = ["skardi/chunking"]
2525
rag = ["embedding", "chunking"]
2626
# documents source connector (mirrors skardi/documents).
2727
documents = ["skardi/documents"]
28+
# rss source connector (mirrors skardi/rss). The typed `rss:` config block on
29+
# `data_sources[]` parses without it; this flag is what makes a `type: rss`
30+
# source register instead of failing with a build-capability error.
31+
rss = ["skardi/rss"]
2832

2933
[dependencies]
3034
better-auth = { workspace = true }

0 commit comments

Comments
 (0)