Skip to content

Commit 737642e

Browse files
KyleAMathewsclaude
KyleAMathews
and
claude
committed
Add Durable Streams 0.1.0 blog post
Announces the first official npm release of Durable Streams with: - npm packages for client, server, state protocol, and conformance tests - Introduction of the State Protocol for database-style sync semantics - Caddy-based production-ready server binary - Comprehensive conformance test improvements (124 server, 110 client tests) - Community experiments and early implementations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent f6e7c75 commit 737642e

File tree

1 file changed

+321
-0
lines changed

1 file changed

+321
-0
lines changed

‎website/blog/posts/2025-12-23-durable-streams-0.1.0.md‎

Lines changed: 321 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,321 @@
1+
---
2+
title: 'Durable Streams 0.1.0: State Protocol, npm Packages, and Early Experiments'
3+
description: >
4+
The first official npm release of Durable Streams, introducing the State Protocol for database-style sync semantics and early community experiments
5+
excerpt: >
6+
Two weeks after announcing Durable Streams, we're shipping 0.1.0 to npm—the first official packages you can install and use in production. This release introduces the State Protocol for database-style sync semantics and includes improved conformance tests.
7+
authors: [kyle, samwillis]
8+
image: /img/blog/announcing-durable-streams/hero.png
9+
tags: [durable-streams, sync, protocol]
10+
outline: [2, 3]
11+
post: true
12+
---
13+
14+
<script setup>
15+
import Tweet from 'vue-tweet'
16+
</script>
17+
18+
Two weeks ago we [announced Durable Streams](https://electric-sql.com/blog/2025/12/09/announcing-durable-streams)—an open protocol for reliable, resumable streaming to client applications. The response has been fantastic: we're approaching 1,000 GitHub stars, and people are already building with it.
19+
20+
Today we're shipping **0.1.0 releases to npm**—the first official packages you can install and use in production. This post covers what's in the release, introduces the State Protocol for database-style sync semantics, and highlights some early experiments from the community.
21+
22+
## What's in 0.1.0
23+
24+
**npm packages:**
25+
26+
```bash
27+
npm install @durable-streams/client # TypeScript client
28+
npm install @durable-streams/server # Reference Node.js server
29+
npm install @durable-streams/state # State Protocol primitives
30+
npm install @durable-streams/server-conformance-tests # Server implementation validation
31+
npm install @durable-streams/client-conformance-tests # Client implementation validation
32+
npm install @durable-streams/cli # Development & testing tools
33+
```
34+
35+
**Server implementations:**
36+
37+
- **Caddy-based server** — A production-ready binary built on Caddy for local development and light production workloads. Download from [GitHub releases](https://github.com/durable-streams/durable-streams/releases) and you're streaming in seconds.
38+
39+
**Client libraries:**
40+
41+
- **TypeScript/JavaScript** — `@durable-streams/client` on npm
42+
- **Go** — `github.com/durable-streams/durable-streams-go`
43+
- **Python** — `pip install durable-streams`
44+
45+
**Conformance test improvements:**
46+
47+
The conformance test suite has grown significantly since launch:
48+
49+
- **124 server conformance tests** validating protocol compliance
50+
- **110 client conformance tests**—entirely new since the announcement post
51+
52+
The client tests are particularly important. We've ported a substantial portion of the test suite from `@electric-sql/client`, which has been battle-tested over 18 months of production use in Electric. These tests cover:
53+
54+
- **Offset semantics**: monotonic offsets, byte-exact resumption without skips or duplicates, offset persistence across sessions
55+
- **Retry behavior**: automatic retry on transient errors (500, 503, 429), respecting Retry-After headers, not retrying permanent errors (4xx)
56+
- **Live streaming**: SSE and long-poll modes, receiving both existing and new data, proper timeout handling, up-to-date signals
57+
- **Streaming equivalence**: verifying SSE and long-poll produce identical results for the same stream and offset
58+
- **Message ordering**: strict order preservation across all read modes (catchup, long-poll, SSE)
59+
- **Producer operations**: stream creation, data appending, batching, sequence ordering
60+
61+
Early feedback from implementers has been invaluable in tightening up the spec and removing ambiguity. Thanks to everyone who's been testing their implementations and reporting edge cases—this kind of community input makes the protocol stronger for everyone.
62+
63+
If you're building your own implementation, both test suites are available as `@durable-streams/server-conformance-tests` and `@durable-streams/client-conformance-tests`. Run them against your server or client to validate compatibility before shipping.
64+
65+
The conformance test suite validates that all implementations behave identically—same protocol, same semantics, your choice of language.
66+
67+
## Try It Out Now
68+
69+
The fastest way to see Durable Streams in action is with the server binary and curl.
70+
71+
**1. Download and run the server:**
72+
73+
Download the latest server binary for your platform from the [releases page](https://github.com/durable-streams/durable-streams/releases/latest).
74+
75+
Available builds: macOS (Intel & ARM), Linux (AMD64 & ARM64), Windows (AMD64).
76+
77+
Extract the archive and run:
78+
79+
```bash
80+
./durable-streams-server
81+
```
82+
83+
The server starts on `http://localhost:8787`.
84+
85+
**2. Create a stream:**
86+
87+
```bash
88+
curl -X PUT http://localhost:8787/v1/stream/my-first-stream
89+
```
90+
91+
**3. Append some data:**
92+
93+
```bash
94+
curl -X POST http://localhost:8787/v1/stream/my-first-stream \
95+
-H "Content-Type: text/plain" \
96+
-d "Hello, Durable Streams!"
97+
```
98+
99+
The response includes an `X-Offset` header—that's your position in the stream.
100+
101+
**4. Read the stream:**
102+
103+
```bash
104+
curl http://localhost:8787/v1/stream/my-first-stream
105+
```
106+
107+
**5. Watch it live:**
108+
109+
Open a terminal and start tailing with SSE:
110+
111+
```bash
112+
curl -N http://localhost:8787/v1/stream/my-first-stream?live=sse
113+
```
114+
115+
In another terminal, append more data:
116+
117+
```bash
118+
curl -X POST http://localhost:8787/v1/stream/my-first-stream \
119+
-H "Content-Type: text/plain" \
120+
-d "This appears in real-time!"
121+
```
122+
123+
Watch it appear instantly in your first terminal. That's durable streaming—ordered, resumable, and live.
124+
125+
## Introducing the State Protocol
126+
127+
In the announcement post, we described a composable ecosystem with Durable Streams as the foundation and higher-level protocols built on top. The **State Protocol** is the first of those higher-level protocols.
128+
129+
Like Durable Streams itself, the State Protocol is extracted from Electric's Postgres sync protocol—refined over 18 months of production use and now standardized as a standalone protocol that works over any durable stream.
130+
131+
Durable Streams gives you ordered, resumable byte delivery. The State Protocol adds semantic meaning: **insert**, **update**, and **delete** operations on typed entities. It's the vocabulary you need for database-style sync—presence tracking, chat rooms, feature flags, collaborative state—without prescribing how you store or query that state.
132+
133+
### The Shape of a Change Event
134+
135+
```typescript
136+
{
137+
type: "user", // Entity type (routes to collection)
138+
key: "user:123", // Unique identifier
139+
value: { // The entity data
140+
name: "Alice",
141+
email: "alice@example.com"
142+
},
143+
headers: {
144+
operation: "insert", // insert | update | delete
145+
txid: "abc-123", // Optional transaction ID
146+
timestamp: "2025-12-23T10:30:00Z"
147+
}
148+
}
149+
```
150+
151+
The protocol also defines control messages (`snapshot-start`, `snapshot-end`, `reset`) for stream lifecycle management.
152+
153+
### Why Separate Protocols?
154+
155+
Separation means you can adopt what you need:
156+
157+
- **AI token streaming?** Use Durable Streams directly—you don't need insert/update/delete semantics for tokens.
158+
- **Real-time database sync?** Add the State Protocol for typed collections with proper CRUD operations.
159+
- **Both in the same app?** Different streams can use different protocols.
160+
161+
### Using the State Protocol
162+
163+
For basic use cases, `MaterializedState` gives you an in-memory key-value store that applies change events:
164+
165+
```typescript
166+
import { MaterializedState } from "@durable-streams/state"
167+
168+
const state = new MaterializedState()
169+
170+
state.apply({
171+
type: "user",
172+
key: "1",
173+
value: { name: "Kyle" },
174+
headers: { operation: "insert" }
175+
})
176+
177+
const user = state.get("user", "1") // { name: "Kyle" }
178+
```
179+
180+
For applications that need reactive queries, filtering, joins, and optimistic updates, `@durable-streams/state` integrates with [TanStack DB](https://tanstack.com/db):
181+
182+
```typescript
183+
import { createStateSchema, createStreamDB } from "@durable-streams/state"
184+
import { useLiveQuery } from "@tanstack/react-db"
185+
import { eq } from "@tanstack/db"
186+
187+
const schema = createStateSchema({
188+
users: {
189+
schema: userSchema,
190+
type: "user",
191+
primaryKey: "id"
192+
}
193+
})
194+
195+
const db = createStreamDB({
196+
streamOptions: { url: streamUrl, contentType: "application/json" },
197+
state: schema
198+
})
199+
200+
// Reactive query that updates automatically
201+
const activeUsers = useLiveQuery((q) =>
202+
q.from({ users: db.collections.users })
203+
.where(({ users }) => eq(users.active, true))
204+
)
205+
```
206+
207+
TanStack DB uses differential dataflow under the hood—queries recompute incrementally when data changes, which is dramatically faster than filtering in JavaScript.
208+
209+
The full State Protocol specification is available at [STATE-PROTOCOL.md](https://github.com/durable-streams/durable-streams/blob/main/packages/state/STATE-PROTOCOL.md).
210+
211+
## Community Experiments
212+
213+
The best part of launching has been seeing what people build. Here's what the community has been exploring:
214+
215+
### AI Agents and Workflows
216+
217+
The agent use case has resonated strongly. Nathan Flurry built an experimental integration combining [Durable Streams with Rivet Actors](https://www.rivet.dev/templates/experimental-durable-streams-ai-agent/)—actors as "the brains & memory" and Durable Streams as "the pipes":
218+
219+
<figure style="background: none">
220+
<Tweet tweet-id="1999512065682423861" conversation="none" theme="dark" />
221+
</figure>
222+
223+
Kames has been building agent workflows with [Mastra](https://mastra.ai/) on top of Durable Streams:
224+
225+
<figure style="background: none">
226+
<Tweet tweet-id="2002776849563431422" conversation="none" theme="dark" />
227+
</figure>
228+
229+
And the conceptual simplicity is clicking for people:
230+
231+
<figure style="background: none">
232+
<Tweet tweet-id="2003530703171195287" conversation="none" theme="dark" />
233+
</figure>
234+
235+
<figure style="background: none">
236+
<Tweet tweet-id="2002666842633220168" conversation="none" theme="dark" />
237+
</figure>
238+
239+
<figure style="background: none">
240+
<Tweet tweet-id="2002187469786083754" conversation="none" theme="dark" />
241+
</figure>
242+
243+
### Resilient Streaming Demos
244+
245+
Sam Willis demonstrated multimodal GenAI streaming (text + audio) with reconnection—lose the connection, reconnect, and keep listening from exactly where you left off while the model keeps generating:
246+
247+
<figure style="background: none">
248+
<Tweet tweet-id="2002037670067806303" conversation="none" theme="dark" />
249+
</figure>
250+
251+
Kyle built demos showing Durable Streams handling the Wikipedia events firehose with TanStack DB and real-time state sync:
252+
253+
<figure style="background: none">
254+
<Tweet tweet-id="2001304555267502499" conversation="none" theme="dark" />
255+
</figure>
256+
257+
<figure style="background: none">
258+
<Tweet tweet-id="2000961535360032845" conversation="none" theme="dark" />
259+
</figure>
260+
261+
### Integration Proposals
262+
263+
The [LiveStore](https://github.com/livestorejs/livestore) team opened [an issue proposing a sync provider](https://github.com/livestorejs/livestore/issues/944) to integrate Durable Streams with their SQLite-powered local-first framework. The discussion explores how the two projects' shared philosophy—append-only event logs, offset-based resumption, local-first architecture—could combine to give users CDN-friendly sync with structured event schemas and reactive UI bindings.
264+
265+
### The Protocol Advantage
266+
267+
People are noting the value of standardization over vendor lock-in:
268+
269+
<figure style="background: none">
270+
<Tweet tweet-id="1999960464990847380" conversation="none" theme="dark" />
271+
</figure>
272+
273+
<figure style="background: none">
274+
<Tweet tweet-id="2001761793740513685" conversation="none" theme="dark" />
275+
</figure>
276+
277+
Even the TanStack team is excited about the HTTP-native approach:
278+
279+
<figure style="background: none">
280+
<Tweet tweet-id="2002063260137500976" conversation="none" theme="dark" />
281+
</figure>
282+
283+
And Nathan Flurry's making bold predictions:
284+
285+
<figure style="background: none">
286+
<Tweet tweet-id="1999217103589769245" conversation="none" theme="dark" />
287+
</figure>
288+
289+
### New Implementations
290+
291+
The protocol is already attracting new implementations. [Ahimsa Labs released a Go client](https://github.com/ahimsalabs/durable-streams-go), and Evil Martians announced they're gradually adopting Durable Streams in [AnyCable](https://anycable.io/)—starting with implementing the read part of the protocol for consuming durable streams. Their post ["AnyCable, Rails, and the pitfalls of LLM-streaming"](https://evilmartians.com/chronicles/anycable-rails-and-the-pitfalls-of-llm-streaming) explores the exact reliability challenges Durable Streams solves:
292+
293+
<figure style="background: none">
294+
<Tweet tweet-id="2001719297651998841" conversation="none" theme="dark" />
295+
</figure>
296+
297+
## What's Next
298+
299+
- **Hosted cloud version**: We're building our own cloud implementation of Durable Streams, launching in early January 2026.
300+
- **More language implementations**: The protocol is designed to have many implementations. We'd love to see servers and clients in Rust, Java, Swift, and more.
301+
- **Database adapters**: Postgres, MySQL, and SQLite adapters using the State Protocol—streaming database changes to clients with proper sync semantics.
302+
- **Electric 2.0**: This is all foundational work for the next version of Electric.
303+
304+
## Get Started
305+
306+
**Install the packages:**
307+
308+
```bash
309+
npm install @durable-streams/client @durable-streams/state
310+
```
311+
312+
**Or use your language of choice:**
313+
314+
```bash
315+
go get github.com/durable-streams/durable-streams-go
316+
pip install durable-streams
317+
```
318+
319+
Run the [test UI](https://github.com/durable-streams/durable-streams/tree/main/packages/test-ui) to experiment locally. If you're building a server implementation, the conformance test suite will validate compatibility.
320+
321+
Join us in [Discord](https://discord.electric-sql.com) to share what you're building.

0 commit comments

Comments
 (0)