Skip to content

Commit 32f026a

Browse files
committed
Rephrase and make CLI more important
1 parent d20f281 commit 32f026a

1 file changed

Lines changed: 62 additions & 51 deletions

File tree

README.md

Lines changed: 62 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33

44
<img src="asset/logo.png" alt="Skardi Logo" width="700">
55

6-
**The Declarative Data Contract Execution Layer for AI and Agents, powered by Rust and Datafusion**<br/>
7-
**Turn SQL queries into live API endpoints — connect CSV, Parquet, PostgreSQL, MySQL, MongoDB, Iceberg, Lance, and S3 in a single query.**
6+
**The Declarative data runtime for AI and agents powered by Rust and Apache Datafusion.**<br/>
7+
**Query files, databases, data lakes, and vector stores with SQL — locally with skardi-cli or as APIs with skardi-server.**
88

99
[CI]: https://github.com/SkardiLabs/skardi/actions/workflows/ci.yml
1010
[CI Badge]: https://github.com/SkardiLabs/skardi/actions/workflows/ci.yml/badge.svg
@@ -16,12 +16,16 @@
1616

1717
<hr />
1818

19-
Skardi lets you define SQL queries in YAML files and instantly serve them as parameterized HTTP APIs. Connect to multiple data sources, run federated queries across them, and expose the results as REST endpoints — all without writing application code.
19+
Skardi lets AI agents and applications query files, databases, data lakes, and vector stores with SQL — no application code required.
20+
21+
- **`skardi-cli`** — Run SQL queries locally against files, object stores, databases, and datalake formats. Ideal for local agents like [OpenClaw](https://github.com/openclaw/openclaw) that need structured data access without a running server.
22+
- **`skardi-server`** — Define SQL queries in YAML and serve them as parameterized HTTP APIs. Connect to multiple data sources, run federated queries, and expose results as REST endpoints.
2023

2124
> **Warning:** This software is in BETA. It may still contain bugs and unexpected behavior. Use caution with production data and ensure you have backups. Feel free to contact us if you want to have a POC for the product.
2225
2326
## Key Features
2427

28+
- **CLI for local agents & queries** — Run SQL against local files, remote object stores (S3, GCS, Azure), databases, and datalake formats — ideal for local AI agents like [OpenClaw](https://github.com/openclaw/openclaw)
2529
- **Declarative pipelines** — Define SQL queries in YAML, get REST APIs automatically
2630
- **Automatic parameter inference** — Request parameters, types, and response schemas are inferred from your SQL
2731
- **Multi-source federation** — JOIN across CSV, Parquet, PostgreSQL, MySQL, MongoDB, Iceberg, and Lance in a single query
@@ -30,12 +34,12 @@ Skardi lets you define SQL queries in YAML files and instantly serve them as par
3034
- **S3 support** — Read CSV, Parquet, and Lance files directly from S3
3135
- **Docker ready** — Ship as a container with your config files mounted at runtime
3236
- **ONNX inference** — Run ONNX model predictions inline in SQL via the `onnx_predict` UDF
33-
- **CLI for local & remote queries** — Run SQL against local files, remote object stores (S3, GCS, Azure), databases, and datalake formats without starting a server
3437

3538
## Table of Contents
3639

3740
- [Quick Start](#quick-start)
3841
- [Architecture](#architecture)
42+
- [Skardi CLI](#skardi-cli)
3943
- [Skardi Server](#skardi-server)
4044
- [Running the Server](#running-the-server)
4145
- [API Endpoints](#api-endpoints)
@@ -44,7 +48,6 @@ Skardi lets you define SQL queries in YAML files and instantly serve them as par
4448
- [In-Memory Caching](#in-memory-caching)
4549
- [Pipeline Files](#pipeline-files)
4650
- [Runtime Pipeline Registration](#runtime-pipeline-registration)
47-
- [Skardi CLI](#skardi-cli)
4851
- [Supported Data Sources](#supported-data-sources)
4952
- [CSV](#csv)
5053
- [Parquet](#parquet)
@@ -65,6 +68,14 @@ Skardi lets you define SQL queries in YAML files and instantly serve them as par
6568
# Build
6669
cargo build --release
6770

71+
# --- Skardi CLI ---
72+
# Query local files directly
73+
skardi query --sql "SELECT * FROM './data/products.csv' LIMIT 10"
74+
75+
# Query remote files
76+
skardi query --sql "SELECT * FROM 's3://mybucket/events.parquet' LIMIT 10"
77+
78+
# --- Skardi Server ---
6879
# Start the server with a context and pipeline
6980
cargo run --bin skardi-server -- \
7081
--ctx demo/ctx.yaml \
@@ -81,11 +92,53 @@ curl -X POST http://localhost:8080/product-search-demo/execute \
8192

8293
Skardi has two main components:
8394

95+
- **`skardi-cli`** (`skardi`) — A command-line tool for running SQL queries against local files, remote object stores, databases, and datalake formats without starting a server. Perfect for powering local AI agents like [OpenClaw](https://github.com/openclaw/openclaw) with structured data access.
8496
- **`skardi-server`** — An HTTP server that loads data sources from a **context file**, registers SQL pipelines, and serves them as REST endpoints.
85-
- **`skardi-cli`** (`skardi`) — A command-line tool for running SQL queries against local files, remote object stores, databases, and datalake formats without starting a server.
8697

8798
Both components use [Apache DataFusion](https://datafusion.apache.org/) as the query engine, which enables federated queries across heterogeneous data sources.
8899

100+
## Skardi CLI
101+
102+
The CLI lets you run SQL queries against local files, remote object stores, databases, and datalake formats — no server required. It's a great fit for local AI agents like [OpenClaw](https://github.com/openclaw/openclaw) that need to query structured data on the fly.
103+
104+
### Install
105+
106+
```bash
107+
cargo install --path crates/cli
108+
```
109+
110+
### Usage
111+
112+
```bash
113+
# Query files directly by path (no context file needed)
114+
skardi query --sql "SELECT * FROM './data/products.csv' LIMIT 10"
115+
skardi query --sql "SELECT * FROM 's3://mybucket/events.parquet' LIMIT 10"
116+
skardi query --sql "SELECT * FROM './embeddings.lance' LIMIT 5"
117+
118+
# Query with a context file (for databases, named tables, etc.)
119+
skardi query --ctx ./ctx.yaml --sql "SELECT * FROM products LIMIT 10"
120+
121+
# SQL from file
122+
skardi query --ctx ./ctx.yaml --file query.sql
123+
124+
# Show table schemas
125+
skardi query --ctx ./ctx.yaml --schema --all
126+
skardi query --ctx ./ctx.yaml --schema -t products
127+
```
128+
129+
**Supported sources:**
130+
131+
| Category | Types |
132+
|----------|-------|
133+
| Local files | CSV, Parquet, JSON/NDJSON, Lance |
134+
| Remote stores | S3, GCS, Azure Blob, HTTP/HTTPS, OSS, COS |
135+
| Datalake formats | Lance, Iceberg |
136+
| Databases | PostgreSQL, MySQL, MongoDB |
137+
138+
**Context file resolution** (when `--ctx` is omitted): checks `SKARDICONFIG` env var, then `~/.skardi/config/ctx.yaml`. If no context file is found, the query runs without pre-registered tables (you can still query files directly by path).
139+
140+
For full details, see [crates/cli/README.md](crates/cli/README.md).
141+
89142
## Skardi Server
90143

91144
### Running the Server
@@ -262,48 +315,6 @@ curl -X POST http://localhost:8080/register_pipeline \
262315
}
263316
```
264317

265-
## Skardi CLI
266-
267-
The CLI lets you run SQL queries against local files, remote object stores, databases, and datalake formats — no server required.
268-
269-
### Install
270-
271-
```bash
272-
cargo install --path crates/cli
273-
```
274-
275-
### Usage
276-
277-
```bash
278-
# Query files directly by path (no context file needed)
279-
skardi query --sql "SELECT * FROM './data/products.csv' LIMIT 10"
280-
skardi query --sql "SELECT * FROM 's3://mybucket/events.parquet' LIMIT 10"
281-
skardi query --sql "SELECT * FROM './embeddings.lance' LIMIT 5"
282-
283-
# Query with a context file (for databases, named tables, etc.)
284-
skardi query --ctx ./ctx.yaml --sql "SELECT * FROM products LIMIT 10"
285-
286-
# SQL from file
287-
skardi query --ctx ./ctx.yaml --file query.sql
288-
289-
# Show table schemas
290-
skardi query --ctx ./ctx.yaml --schema --all
291-
skardi query --ctx ./ctx.yaml --schema -t products
292-
```
293-
294-
**Supported sources:**
295-
296-
| Category | Types |
297-
|----------|-------|
298-
| Local files | CSV, Parquet, JSON/NDJSON, Lance |
299-
| Remote stores | S3, GCS, Azure Blob, HTTP/HTTPS, OSS, COS |
300-
| Datalake formats | Lance, Iceberg |
301-
| Databases | PostgreSQL, MySQL, MongoDB |
302-
303-
**Context file resolution** (when `--ctx` is omitted): checks `SKARDICONFIG` env var, then `~/.skardi/config/ctx.yaml`. If no context file is found, the query runs without pre-registered tables (you can still query files directly by path).
304-
305-
For full details, see [crates/cli/README.md](crates/cli/README.md).
306-
307318
## Supported Data Sources
308319

309320
### CSV
@@ -577,14 +588,14 @@ docker run --rm -p 8080:8080 skardi
577588
git clone https://github.com/SkardiLabs/skardi.git
578589
cd skardi
579590
580-
# Build server
581-
cargo build --release -p skardi-server
582-
583591
# Build CLI
584592
cargo build --release -p skardi-cli
585593
586594
# Or install CLI globally
587595
cargo install --path crates/cli
596+
597+
# Build server
598+
cargo build --release -p skardi-server
588599
```
589600

590601
## Demo & Examples

0 commit comments

Comments
 (0)