federated-queries.md

Federated Queries

One of Skardi's most powerful features is the ability to JOIN data across different source types in a single SQL query. DataFusion handles the federation transparently.

Example: CSV + PostgreSQL

Join a CSV file with a PostgreSQL table and write results back to PostgreSQL:

kind: pipeline

metadata:
  name: "federated_join_and_insert"
  version: "1.0"

spec:
  query: |
    INSERT INTO user_order_stats (user_id, user_name, total_orders, total_spent)
    SELECT
      u.id as user_id,
      u.name as user_name,
      COUNT(o.order_id) as total_orders,
      SUM(o.amount) as total_spent
    FROM users u                    -- PostgreSQL table
    INNER JOIN csv_orders o         -- CSV file
      ON u.id = o.user_id
    WHERE u.name = {name}
    GROUP BY u.id, u.name

More Examples

Federated query examples are included in each data source's docs:

• PostgreSQL — pipelines/federated_join_and_insert.yaml
• MySQL — pipelines/federated_join_and_insert.yaml
• SQLite — pipelines/federated_join_and_insert.yaml
• Redis — pipelines/federated_join_and_insert.yaml