Skip to content

Commit 39117e0

Browse files
committed
fix(pulse-core): reference-count shared subscriptions
`subscribe()` and `subscribeContract()` memoise by key, so concurrent callers asking for the same address or contract receive the *same* Watcher object. `unsubscribe()` called `stop()` on it unconditionally. `Watcher.stop()` sets `_stopped`, calls `removeAllListeners()`, and makes `emit()` return false without dispatching — so the first caller to leave silently killed every other caller's event flow. The failure is invisible from the outside. In apps/web both SSE routes key on data that is not per-connection (`address`, `contract:${contractId}`) and unsubscribe on teardown, so when one visitor closed their tab, every other visitor watching the same contract kept an open connection, kept receiving heartbeats, and never received another event — indistinguishable from a quiet contract. Adds a namespaced refcount map (`addr:` / `contract:` / `config:`, since the three registries have independent key spaces). `subscribe*()` retains, `unsubscribe*()` releases, and the watcher stops only on the last release. Details worth noting: - Each watcher's stop handler clears its own refcount entry, so a consumer calling `watcher.stop()` directly cannot strand a count and leave the next subscription for that key permanently unstoppable. - `release()` on an unknown key returns true (stop). An already-torn-down entry should not keep a watcher alive. - `unsubscribeAll()` / `unsubscribeAllContracts()` are teardown and deliberately ignore counts; `unsubscribeAll()` now iterates a snapshot because the stop handlers mutate the registry it was iterating. Single-subscriber behaviour is unchanged: one subscribe, one unsubscribe, stopped immediately. Verified end to end against a live server: two clients streaming mainnet USDC (CCW67TSZ...), first disconnected, second went on receiving real `contract.emitted` events rather than going silent. 12 new tests; all 603 existing pulse-core tests still pass.
1 parent 3ce8c59 commit 39117e0

2 files changed

Lines changed: 253 additions & 7 deletions

File tree

packages/pulse-core/src/EventEngine.ts

Lines changed: 88 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,13 @@ function stableFilterKey(filters: ContractFilter[]): string {
143143
* The Date is parsed from `event.timestamp` on first access and cached.
144144
* JSON.stringify output is unaffected because the property is non-enumerable.
145145
*/
146+
/** Namespaced refcount keys - the three subscription registries key independently. */
147+
const refKey = {
148+
address: (address: string) => `addr:${address}`,
149+
contract: (id: string) => `contract:${id}`,
150+
config: (filterKey: string) => `config:${filterKey}`,
151+
};
152+
146153
function withTimestampDate<T extends { timestamp: string }>(event: T): Timestamped<T> {
147154
let cached: Date | undefined;
148155
Object.defineProperty(event, "timestampDate", {
@@ -170,6 +177,20 @@ export class EventEngine {
170177
* `unsubscribeContract(config)` for lookup.
171178
*/
172179
private contractConfigRegistry: Map<string, Watcher> = new Map();
180+
/**
181+
* How many outstanding `subscribe*()` calls share each registry entry.
182+
*
183+
* Subscriptions are memoised by key, so concurrent callers asking for the
184+
* same address or contract get the *same* `Watcher` object. Without a count,
185+
* the first `unsubscribe()` would call `stop()` on that shared watcher and
186+
* silently kill every other caller's event flow - `stop()` removes all
187+
* listeners and makes `emit()` a no-op, so the others keep their connection
188+
* open and simply never receive anything again.
189+
*
190+
* Keys are namespaced (`addr:` / `contract:` / `config:`) because the three
191+
* registries have independent key spaces that could otherwise collide.
192+
*/
193+
private refCounts: Map<string, number> = new Map();
173194
private subscriptionNames: Map<string, string> = new Map();
174195
private stopStream: HorizonStreamStopper | null = null;
175196
private reconnectTimer: ReturnType<typeof setTimeout> | null = null;
@@ -511,16 +532,43 @@ export class EventEngine {
511532
}
512533
}
513534

535+
/** Records one more holder of a shared subscription. */
536+
private retain(key: string): void {
537+
this.refCounts.set(key, (this.refCounts.get(key) ?? 0) + 1);
538+
}
539+
540+
/**
541+
* Drops one holder of a shared subscription.
542+
* @returns true when that was the last holder and the watcher should stop.
543+
*/
544+
private release(key: string): boolean {
545+
const count = this.refCounts.get(key);
546+
// Unknown key means the watcher was already torn down (or stopped directly
547+
// via `watcher.stop()`); treat it as the final release.
548+
if (count === undefined) return true;
549+
if (count <= 1) {
550+
this.refCounts.delete(key);
551+
return true;
552+
}
553+
this.refCounts.set(key, count - 1);
554+
return false;
555+
}
556+
514557
/**
515558
* Subscribes to events for a given Stellar address.
516559
* Returns an existing Watcher if one already exists for the address.
560+
*
561+
* The returned Watcher is shared between callers asking for the same
562+
* address. It stays alive until every caller has unsubscribed, so one
563+
* caller disconnecting cannot silence the others.
517564
* @param address - The Stellar address to watch.
518565
* @param options - Optional subscription options, including a filter predicate.
519566
* @returns The Watcher instance for the address.
520567
*/
521568
subscribe(address: string, options?: SubscribeOptions): Watcher {
522569
const existingWatcher = this.registry.get(address);
523570
if (existingWatcher) {
571+
this.retain(refKey.address(address));
524572
if (options?.filter) {
525573
const name = this.subscriptionNames.get(address);
526574
if (name !== undefined) {
@@ -539,6 +587,7 @@ export class EventEngine {
539587
}
540588

541589
const watcher = new Watcher(address);
590+
this.retain(refKey.address(address));
542591
if (options?.name !== undefined) {
543592
this.subscriptionNames.set(address, options.name);
544593
}
@@ -554,6 +603,7 @@ export class EventEngine {
554603
}
555604
watcher.addStopHandler(() => {
556605
this.registry.delete(address);
606+
this.refCounts.delete(refKey.address(address));
557607
this.subscriptionNames.delete(address);
558608
for (const subWatcher of subWatchers) subWatcher.stop();
559609
});
@@ -566,6 +616,7 @@ export class EventEngine {
566616
}
567617
watcher.addStopHandler(() => {
568618
this.registry.delete(address);
619+
this.refCounts.delete(refKey.address(address));
569620
this.filters.delete(address);
570621
this.subscriptionNames.delete(address);
571622
});
@@ -574,19 +625,32 @@ export class EventEngine {
574625
}
575626

576627
/**
577-
* Unsubscribes from events for a given Stellar address and stops its watcher.
628+
* Releases one subscription to a given Stellar address.
629+
*
630+
* The underlying Watcher is shared, so it is only stopped once every caller
631+
* that subscribed to this address has unsubscribed. Stopping it while another
632+
* caller still holds it would remove their listeners and silently end their
633+
* event flow.
578634
* @param address - The Stellar address to stop watching.
579635
*/
580636
unsubscribe(address: string): void {
581-
this.registry.get(address)?.stop();
637+
const watcher = this.registry.get(address);
638+
if (!watcher) return;
639+
if (this.release(refKey.address(address))) {
640+
watcher.stop();
641+
}
582642
}
583643

584644
/**
585645
* Stops all active watchers without closing the underlying SSE stream.
586646
* Use this to drain subscriptions while keeping the stream open.
647+
*
648+
* This is a teardown operation and deliberately ignores reference counts:
649+
* it stops every watcher outright, however many holders each has. Each
650+
* watcher's stop handler clears its own refcount entry.
587651
*/
588652
unsubscribeAll(): void {
589-
for (const watcher of this.registry.values()) {
653+
for (const watcher of [...this.registry.values()]) {
590654
watcher.stop();
591655
}
592656
}
@@ -625,9 +689,13 @@ export class EventEngine {
625689

626690
const key = stableFilterKey(config.filters);
627691
const existing = this.contractConfigRegistry.get(key);
628-
if (existing) return existing;
692+
if (existing) {
693+
this.retain(refKey.config(key));
694+
return existing;
695+
}
629696

630697
const watcher = new Watcher(key);
698+
this.retain(refKey.config(key));
631699

632700
if (this.networkSources) {
633701
const subWatchers: Watcher[] = [];
@@ -638,13 +706,17 @@ export class EventEngine {
638706
}
639707
watcher.addStopHandler(() => {
640708
this.contractConfigRegistry.delete(key);
709+
this.refCounts.delete(refKey.config(key));
641710
for (const subWatcher of subWatchers) subWatcher.stop();
642711
});
643712
this.contractConfigRegistry.set(key, watcher);
644713
return watcher;
645714
}
646715

647-
watcher.addStopHandler(() => this.contractConfigRegistry.delete(key));
716+
watcher.addStopHandler(() => {
717+
this.contractConfigRegistry.delete(key);
718+
this.refCounts.delete(refKey.config(key));
719+
});
648720
this.contractConfigRegistry.set(key, watcher);
649721
return watcher;
650722
}
@@ -653,6 +725,7 @@ export class EventEngine {
653725
const id = idOrConfig;
654726
const existing = this.contractRegistry.get(id);
655727
if (existing) {
728+
this.retain(refKey.contract(id));
656729
if (options?.filter) {
657730
this.log.warn(
658731
`[pulse-core] subscribeContract() called for ${this.describeSubscription(id)} which already has an active watcher - filter option ignored.`,
@@ -663,6 +736,7 @@ export class EventEngine {
663736
}
664737

665738
const watcher = new Watcher(id);
739+
this.retain(refKey.contract(id));
666740
const filters = options?.filters ?? [];
667741
if (options?.name !== undefined) {
668742
this.subscriptionNames.set(id, options.name);
@@ -677,6 +751,7 @@ export class EventEngine {
677751
}
678752
watcher.addStopHandler(() => {
679753
this.contractRegistry.delete(id);
754+
this.refCounts.delete(refKey.contract(id));
680755
this.subscriptionNames.delete(id);
681756
for (const subWatcher of subWatchers) subWatcher.stop();
682757
});
@@ -689,6 +764,7 @@ export class EventEngine {
689764
}
690765
watcher.addStopHandler(() => {
691766
this.contractRegistry.delete(id);
767+
this.refCounts.delete(refKey.contract(id));
692768
this.subscriptionNames.delete(id);
693769
this.filters.delete(id);
694770
if (this.contractRegistry.size === 0 && this.sorobanSubscriber) {
@@ -717,11 +793,16 @@ export class EventEngine {
717793
unsubscribeContract(idOrConfig: string | ContractSubscriptionConfig): void {
718794
if (typeof idOrConfig === "object") {
719795
const key = stableFilterKey(idOrConfig.filters);
796+
const watcher = this.contractConfigRegistry.get(key);
797+
if (!watcher) return;
720798
// The watcher's stop handler removes it from contractConfigRegistry.
721-
this.contractConfigRegistry.get(key)?.stop();
799+
// Only the last holder may stop it - see `refCounts`.
800+
if (this.release(refKey.config(key))) watcher.stop();
722801
return;
723802
}
724-
this.contractRegistry.get(idOrConfig)?.watcher.stop();
803+
const entry = this.contractRegistry.get(idOrConfig);
804+
if (!entry) return;
805+
if (this.release(refKey.contract(idOrConfig))) entry.watcher.stop();
725806
}
726807

727808
/**
Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
import { describe, it, expect } from "vitest";
2+
import { EventEngine } from "../src/EventEngine.js";
3+
4+
const ADDRESS = "GABCDEFGHIJKLMNOPQRSTUVWXYZ234567ABCDEFGHIJKLMNOPQRSTUV";
5+
const CONTRACT = "CCW67TSZV3SSS2HXMBQ5JFGCKJNXKZM7UQUWUZPUTHXSTZLEO7SJMI75";
6+
7+
function engine(): EventEngine {
8+
return new EventEngine({ horizonUrl: "https://horizon-testnet.stellar.org" });
9+
}
10+
11+
/** Counts events a subscriber actually receives. */
12+
function collect(watcher: ReturnType<EventEngine["subscribe"]>, eventType: string): unknown[] {
13+
const seen: unknown[] = [];
14+
watcher.on(eventType, (event) => seen.push(event));
15+
return seen;
16+
}
17+
18+
describe("subscribe() reference counting", () => {
19+
it("hands concurrent callers the same shared Watcher", () => {
20+
const e = engine();
21+
expect(e.subscribe(ADDRESS)).toBe(e.subscribe(ADDRESS));
22+
});
23+
24+
it("regression (MEDIUM-4): one caller unsubscribing does not silence the others", () => {
25+
// Two HTTP clients streaming the same address. Before ref counting, client
26+
// A's teardown called stop() on the watcher they *shared*, which removed
27+
// B's listeners and made emit() a no-op - B's connection stayed open and
28+
// simply never delivered another event.
29+
const e = engine();
30+
const a = e.subscribe(ADDRESS);
31+
const b = e.subscribe(ADDRESS);
32+
const bReceived = collect(b, "payment.received");
33+
34+
e.unsubscribe(ADDRESS); // client A disconnects
35+
36+
expect(b.stopped).toBe(false);
37+
expect(a.stopped).toBe(false);
38+
b.emit("payment.received", { seq: 1 });
39+
expect(bReceived).toHaveLength(1);
40+
});
41+
42+
it("stops the watcher once the last caller unsubscribes", () => {
43+
const e = engine();
44+
const w = e.subscribe(ADDRESS);
45+
e.subscribe(ADDRESS);
46+
47+
e.unsubscribe(ADDRESS);
48+
expect(w.stopped).toBe(false);
49+
50+
e.unsubscribe(ADDRESS);
51+
expect(w.stopped).toBe(true);
52+
});
53+
54+
it("still stops immediately for a single subscriber", () => {
55+
const e = engine();
56+
const w = e.subscribe(ADDRESS);
57+
e.unsubscribe(ADDRESS);
58+
expect(w.stopped).toBe(true);
59+
});
60+
61+
it("does not carry a stale count into a fresh subscription", () => {
62+
const e = engine();
63+
const first = e.subscribe(ADDRESS);
64+
e.subscribe(ADDRESS);
65+
e.unsubscribe(ADDRESS);
66+
e.unsubscribe(ADDRESS);
67+
expect(first.stopped).toBe(true);
68+
69+
// A new subscription for the same address must start from zero, not
70+
// inherit the previous entry's count.
71+
const second = e.subscribe(ADDRESS);
72+
expect(second).not.toBe(first);
73+
e.unsubscribe(ADDRESS);
74+
expect(second.stopped).toBe(true);
75+
});
76+
77+
it("tolerates unsubscribing more times than subscribed", () => {
78+
const e = engine();
79+
const w = e.subscribe(ADDRESS);
80+
e.unsubscribe(ADDRESS);
81+
expect(() => e.unsubscribe(ADDRESS)).not.toThrow();
82+
expect(w.stopped).toBe(true);
83+
});
84+
85+
it("keeps counts independent per address", () => {
86+
const other = "GBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
87+
const e = engine();
88+
const a = e.subscribe(ADDRESS);
89+
e.subscribe(ADDRESS);
90+
const b = e.subscribe(other);
91+
92+
e.unsubscribe(other);
93+
expect(b.stopped).toBe(true);
94+
expect(a.stopped).toBe(false);
95+
});
96+
97+
it("unsubscribeAll tears down regardless of outstanding holders", () => {
98+
const e = engine();
99+
const w = e.subscribe(ADDRESS);
100+
e.subscribe(ADDRESS);
101+
e.subscribe(ADDRESS);
102+
103+
e.unsubscribeAll();
104+
expect(w.stopped).toBe(true);
105+
106+
// The refcount entry must be gone too, or the next subscription would
107+
// start life already "held" and never stop.
108+
const fresh = e.subscribe(ADDRESS);
109+
e.unsubscribe(ADDRESS);
110+
expect(fresh.stopped).toBe(true);
111+
});
112+
113+
it("clears counts when a watcher is stopped directly, bypassing unsubscribe", () => {
114+
const e = engine();
115+
const w = e.subscribe(ADDRESS);
116+
e.subscribe(ADDRESS);
117+
118+
w.stop(); // consumer stopped the Watcher itself
119+
120+
const fresh = e.subscribe(ADDRESS);
121+
expect(fresh).not.toBe(w);
122+
e.unsubscribe(ADDRESS);
123+
expect(fresh.stopped).toBe(true);
124+
});
125+
});
126+
127+
describe("subscribeContract() reference counting", () => {
128+
it("regression (MEDIUM-4): shared contract watchers survive one caller leaving", () => {
129+
// Mirrors /api/contracts/[contractId], which keys every connection on the
130+
// same `contract:${contractId}` subscription id.
131+
const e = engine();
132+
const a = e.subscribeContract(`contract:${CONTRACT}`);
133+
const b = e.subscribeContract(`contract:${CONTRACT}`);
134+
expect(a).toBe(b);
135+
136+
e.unsubscribeContract(`contract:${CONTRACT}`);
137+
expect(b.stopped).toBe(false);
138+
139+
e.unsubscribeContract(`contract:${CONTRACT}`);
140+
expect(b.stopped).toBe(true);
141+
});
142+
143+
it("reference counts config-based subscriptions by filter key", () => {
144+
const e = engine();
145+
const config = { filters: [{ contractIds: [CONTRACT] }] };
146+
const a = e.subscribeContract(config);
147+
const b = e.subscribeContract({ filters: [{ contractIds: [CONTRACT] }] });
148+
expect(a).toBe(b); // same canonical filter key
149+
150+
e.unsubscribeContract(config);
151+
expect(a.stopped).toBe(false);
152+
153+
e.unsubscribeContract(config);
154+
expect(a.stopped).toBe(true);
155+
});
156+
157+
it("unsubscribeAllContracts tears down regardless of holders", () => {
158+
const e = engine();
159+
const w = e.subscribeContract(`contract:${CONTRACT}`);
160+
e.subscribeContract(`contract:${CONTRACT}`);
161+
162+
e.unsubscribeAllContracts();
163+
expect(w.stopped).toBe(true);
164+
});
165+
});

0 commit comments

Comments
 (0)