bStocks: Binance-sourced synthetic markets in /v2/rates - #39
bStocks: Binance-sourced synthetic markets in /v2/rates#39stultusmundi wants to merge 9 commits into
Conversation
Adds a Binance singleton provider (mirroring CoinGecko's shape) that fetches the tokenised bStocks universe and Spot 24h/7d tickers from Binance's public endpoints, no API key required. Exposes getTokenisedAssets (1h cache, BSC-only filter), getTradingSymbols (1h cache, TRADING-status filter from exchangeInfo), getTicker24h (60s cache, single batched call), and getTicker7d (60s cache, chunked to 20 symbols/request via the existing arraySplit util). Ticker fetches keep a last-known-good value per symbol so a halted/omitted symbol (e.g. during a stock-split trading break) or an outright failed refresh still serves the previous price instead of dropping the entry. Fixtures are built from live curls of both Binance endpoints rather than assumed shapes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Binance does not omit a halted symbol from the ticker response - it returns the symbol present with lastPrice "0.00000000". Measured live: of 20 BREAK-status symbols requested, 20 came back present and 9 were priced at zero. So the absent-symbol fallback was dead code for the exact scenario it was written for, and the zero was accepted as fresh AND written to the last-known-good store - poisoning it, so a later outage would serve $0 forever rather than the real last price. mergeTickers now admits only finite, strictly positive prices, both to the served set and to the store. Adds lastGoodAgeMs() so a caller can tell a live price from one carried through a long halt, and sorts the ticker cache keys so the same request in a different symbol order still hits cache. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Intersects the Binance tokenised-asset universe (BSC-listed, from Task 1) with TRADING Spot symbols to emit CryptoPrice[] entries for bStocks, quoted against BTCUSDT fetched in the same batch so both legs share one venue. Ids are bstock-<code> under provider "coingecko" per the cross-repo id contract with the sibling api repo. A module-level last-known-good map keeps serving a symbol's previous price across a refresh where it's omitted (CEX halt around a stock split), rather than dropping it. Verified against live Binance endpoints: 66 tokenised assets are all BSC-listed, and intersecting with exchangeInfo's TRADING Spot symbols yields 56/66 tradable today (the other 10, e.g. NFLXB/ASMLB, aren't listed on Spot at all yet) — matching the brief's expected ~56/66. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
… path
The guard against a missing or zero BTCUSDT was correct but untested, so a
later simplification could silently put Infinity or NaN into rates.btc with
nothing in CI to catch it - the same shape of bug that reached a signing path
in the sibling api repo. Mutation-tested: dropping the btcUsd half of the
guard fails both new tests.
Also pins that one halted asset does not affect its siblings in the same
batch, and that the feature flag actually suppresses output.
Corrects two JSDoc claims that came from the plan text: the client does no
provider-prefix parsing (applyMarkets and use-fiat.js each build the literal
`${provider}-${id}` and the strings simply have to match), and Binance does
not omit halted symbols - it returns them present at zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Adds a fifth try/catch provider block to zelcoreRatesV2.getAll() that
appends getBstockPrices() (Task 2) into the crypto array, wrapped so a
Binance/bStocks outage sets errors.binance = true instead of throwing out
of getAll(). Also excludes synthetic `bstock-*` ids from the CoinGecko
harvest in coinAggregatorIDs.ts so rates-api never queries CoinGecko for
them.
Fixes a pre-existing positional-merge bug in apiServices.ts: the crypto
refresh merge used mergeDeep, which walks target/source arrays by index.
Since `processed` in zelcoreRatesV2.getAll() is a concatenation of four
independent try/catch provider blocks, its length and per-index identity
shift between refresh cycles whenever any block throws or an upstream API
returns a different row count -- both routine occurrences. Concretely: if
CoinGecko's block (rank/change7d present) fails one cycle while
CryptoCompare's block (no rank/change7d) succeeds, mergeDeep deep-merges
the old CoinGecko entry at index 0 with the new CryptoCompare entry at
index 0 -- id/provider/rates get correctly overwritten, but `rank` survives
from the stale CoinGecko entry, producing a CryptoCompare coin wearing a
foreign coin's rank. Separately, if the new array is shorter than the old
one, mergeDeep's source.forEach never visits the trailing old indices, so
entries missing from the new fetch persist in the output forever with
frozen data instead of being dropped.
Replaces the crypto merge with mergeCryptoByKey, which discards the stale
target and rebuilds the array from source only, keyed by
`${provider}-${id}`. tests/mergeCrypto.spec.ts pins this: one test
documents mergeDeep's corruption (still used unchanged for fiat/rates/
marketsUSD) on realistic shape-mismatched input, one confirms
mergeCryptoByKey fixes it, and one confirms mergeCryptoByKey produces
identical output to mergeDeep for a normal, well-ordered refresh so
existing consumers see no behavioural change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
…merge key The crypto.length > 300 sanity floor was calibrated before bStocks existed. With ~56 synthetic entries added, a degraded CoinGecko response (250 rows instead of ~342, which it returns silently rather than throwing) now clears the floor where it used to be blocked: 250+39+2 = 291 was rejected, 250+39+2+56 = 347 is accepted. Combined with the wholesale replace, that truncates /v2/rates and drops ~100 coins for the cycle. The floor now counts real-provider rows only. mergeCryptoByKey is renamed replaceCryptoByKey and its unused _target parameter dropped. It never merged - it discards target entirely - and the old signature invited a maintainer to wire target back in, which would throw on the first refresh cycle where ratesV2.crypto is genuinely undefined. A mutation run showed a bare spread of source passed every test in the file: nothing pinned that provider is part of the key, that duplicates collapse, or that the last write wins. Two tests now cover those; all three surviving mutants are killed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Documents the synthetic Binance bStock entries in /v2/rates: universe selection (BSC contract + TRADING Spot symbol), USDT quoting, the provider:"coingecko" id contract with the client and the sibling api repo, and the halted-symbol/last-known-good behavior. Regenerates the committed docs/ typedoc tree to pick up the new bstocks/binance modules and types. Bumped the typedoc devDependency ^0.26.7 -> ^0.28.0: typedoc-plugin- markdown@4.12 (already the installed version, satisfying the existing ^4.2.7 range) requires typedoc 0.28.x as a peer, so `npx typedoc` - the docs-refresh step this same README documents - was silently broken on a clean install before this change. Dev-only, no runtime effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
…ry-forward, outage visibility, refresh bounding Addresses the whole-branch review of bstocks (PR #39): 1. binance.ts: lastGoodTicker was keyed on bare symbol, so getTicker24h and getTicker7d shared one fallback slot. Whichever window last wrote silently overwrote the other's priceChangePercent/quoteVolume, so a 24h fallback could serve the 7-day change (and ~7x-inflated volume) as the 24-hour figure. Now keyed per `${window}:${symbol}`; mergeTickers and lastGoodAgeMs take an explicit window. 2. apiServices.ts: serviceRefresher rebuilt ratesV2.crypto from the fresh fetch alone once the >300-row floor passed. A provider whose block failed this cycle (CryptoCompare's ~39 rows, LiveCoinWatch's 2) would vanish from /v2/rates outright whenever the remaining providers alone still cleared the floor -- new in this branch, since replaceCryptoByKey has no positional stale tail to fall back on. Now carries forward only the rows belonging to providers present in ratesV2Fetched.errors, then key-merges with fresh data last so fresh always wins. 3. bstocks.ts / zelcoreRatesV2.ts: every failure inside getBstockPrices is caught internally, so a total Binance outage looked identical to a healthy refresh while quietly re-serving frozen prices forever. Added isBstocksDegraded() (true when a refresh prices zero symbols fresh while last-known-good is non-empty), wired into errors.binance, and bounded last-known-good to config.bstocksLastGoodMaxAgeMs (7 days, chosen because the outage this exists for -- a stock split halt -- is naturally multi-day). 4. zelcoreRatesV2.ts / binance.ts: a Binance outage cost up to ~92s (AxiosWrapper's retry budget across getTokenisedAssets, getTradingSymbols, and both ticker windows), stalling the refresh of all other providers behind it every 30s cycle. getBstockPrices() is now raced against a 10s timeout, and getTokenisedAssets/ getTradingSymbols negatively-cache a failure for config.binanceFailureCacheMs (60s) so the retry storm doesn't repeat every cycle. The timeout's setTimeout is .unref()'d since Promise.race never cancels the losing branch. 5. config/index.ts: bStocksEnabled now reads process.env.BSTOCKS_ENABLED !== 'false' instead of a literal, so it can be disabled without a redeploy. 6. bstocks.ts: rank is no longer hardcoded to 0 (which sorted every bStock ahead of Bitcoin in ascending rank order) -- omitted, matching the convention CryptoCompare's rows already use elsewhere in this repo. 7. New test coverage: serviceRefresher (previously zero coverage) for the small-provider-outage guard/carry-forward interaction; a total-outage integration case pinning that getBstockPrices never rejects; and a multi-asset property assertion over the coingecko/bstock-<code> cross-repo id contract. Each fix has a test that was verified to fail before the change and pass after (confirmed via targeted git-stash reverts per file). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Fixes from final whole-branch reviewThis commit (87b3888) addresses all 6 findings from the final review, plus the 3 test-coverage blind spots called out alongside them. 1. 24h/7d last-known-good collision ( 2. Coins vanishing on a small-provider outage ( 3. Invisible total Binance outage + unbounded staleness ( 4. Binance outage slowing the whole refresh cycle ( 5. No env kill switch ( 6. Test blind spots closed: Verification: 🤖 Generated with Claude Code |
…hing The 10s bound added for the slow-refresh finding resolved to [] on timeout, which reintroduced the vanishing-rows failure it was meant to prevent, for bStocks specifically. The provider carry-forward in apiServices keys on the row's provider, but bStock rows carry provider "coingecko" while their failure is reported under errors.binance - so nothing could ever carry them. On a hung Binance the first cycle produced zero bStock rows AND an empty errors object: the wallet shows $0 with no banner. Since AxiosWrapper's worst case is ~23s against a 10s bound, the very first cycle of any outage hits it, and rows then blink out roughly every third cycle as the 60s negative cache expires. The timeout branch now serves the pruned last-known-good snapshot and flags the cycle degraded, since freshPricedLastRun still reflects an earlier run when the race is lost. isBstocksDegraded no longer requires lastGood to be non-empty: a cold start during an outage had nothing cached, so it reported a healthy service that was serving no bStocks at all. It now returns false when the feature is disabled, so "off" stays distinguishable from "broken". Also makes BSTOCKS_ENABLED case-insensitive - BSTOCKS_ENABLED=FALSE silently left the feature on - and documents it in .env.example, where an operator can actually find it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Summary
Adds Binance bStocks (tokenized US equities on BNB Smart Chain) as synthetic
market entries in
GET /v2/rates, sourced live from Binance's public SpotAPI. Design spec:
docs/superpowers/specs/2026-08-04-bstocks-integration-design.mdin the ZelTreZ repo (§4.2 covers this repo's contract).
Binanceprovider client (src/services/providers/binance.ts) —singleton, no API key, tokenised-asset list + Spot 24h/7d tickers, with a
last-known-good ticker cache.
getBstockPrices()assembler (src/services/bstocks.ts) — intersectsthe BSC-listed tokenised-asset universe with
TRADING-status Spot symbols,quoted in USDT (verified live — no USDC pairs exist), converts to BTC
via the same ticker batch's
BTCUSDT.zelcoreRatesV2.getAll()and served through the existing/v2/ratesroute.Emitted ids:
bstock-<assetCode lowercase>(e.g.bstock-tslab), alwaystagged
provider: "coingecko"— not"binance". The ZelCore client doesno provider-prefix parsing; it keys its market store on the literal string
${provider}-${id}, and the siblingapirepo advertises each bStock'scoinInfo.coingeckoIDasbstock-<code>. The two literals only meet if theprovider here is exactly
"coingecko"— any other value misses silently,with no error. This id/provider pairing is a cross-repo contract; do not
change it in isolation.
Please read before reviewing the diff
1. The positional-merge fix is the highest-impact change in this PR, and it
affects every provider, not just bStocks. The old
mergeDeepinapiServices.ts'sserviceRefresheroverlaid the freshly-fetchedcryptoarray onto the previous one by array index. That's only correct while
every provider block returns exactly the same row count in the same order.
When a block shrank (a provider outage, a delisted coin), two things broke:
fields from the old entry at that index leaked onto a different coin (e.g.
a CryptoCompare row inheriting CoinGecko's stale
rank/change7d), andentries past the new, shorter length lived on as stale duplicates. Because
the ZelCore client re-keys on
${provider}-${id}with last-write-wins, andthose stale duplicates sat after the fresh ones in the array, wallet users
were served the stale price on any refresh cycle where a provider
block's row count shifted. Reproduced with a concrete input (simulated
CryptoCompare outage: 40 duplicate keys, client resolved to the stale price
instead of the fresh one). Fixed by
replaceCryptoByKey— a key-basedrebuild from the fresh fetch alone (dedup by
${provider}-${id},last-write-wins, no merge with the previous array). Two behaviour changes
worth knowing: repeated
provider+identries collapse to one at the firstoccurrence's position, and an entry the fetch no longer produces disappears
immediately instead of persisting from the previous cycle.
2. The
crypto.length > 300degraded-response guard now counts non-bStockrows only. That floor exists to reject an under-strength provider response
before it overwrites good in-memory data. It was calibrated before bStocks
existed; the ~56 synthetic bStock rows would otherwise pad the count and let
a genuinely degraded CoinGecko/CryptoCompare/LiveCoinWatch response sail
past the guard, truncating
/v2/ratesfor real assets.3. Binance returns a halted symbol present, not absent, during a trading
halt (e.g. around a stock split) — with
lastPrice: "0.00000000". Verifiedlive: 20/20 requested BREAK-status symbols came back present in the ticker
response, 9/20 of those priced at zero. An absent-symbol fallback would have
been dead code for this case, and accepting the zero would both serve a $0
price and poison the last-known-good cache. Prices are therefore accepted
only when finite and strictly positive (
Number.isFinite(px) && px > 0),both for what's served and for what's written to the last-known-good store.
4.
npm run lintis a no-op repo-wide, unrelated to this branch.npx eslint ./ --format jsonlints exactly one file,.eslintrc.jsitself,and zero
.tsfiles —.eslintrc.jsusesparserOptions.parser: babel-eslintwith no TypeScript parser, and ESLint defaults to.jsonly.Every "lint clean" claim in this repo's history is vacuously true.
npx tsc --noEmitis the only real static check and is clean on thisbranch. Flagging as a follow-up; not something this PR fixes.
5. Pre-existing debt this PR does not fix:
mergeDeepis stillpositional for
ratesV2.fiat, v1rates, andmarketsUSDinapiServices.ts— the same stale-tail class of bug described in point 1,just not yet observed to matter for those paths (v1
rates/marketsUSDarevalue-not-array shaped per currency code, so the failure mode is different in
practice;
ratesV2.fiatis a small, rarely-outaged list from a singleprovider). Worth its own ticket rather than folding into this change.
Live verification (real, not simulated)
No automated test proves bStock entries actually reach
/v2/ratesagainstreal Binance, so I booted the service locally and hit the live endpoint.
Environment note: this sandbox has no paid CoinGecko Pro / CryptoCompare /
LiveCoinWatch API keys, and
express-prometheus-middleware'sprom-clientpeer dependency is missing from
package.json(pre-existing, unrelated tothis branch — flagging separately, not fixed here). To get a real listening
server for this verification I temporarily (1) installed
prom-clientlocally and (2) pointed
config.coinGeckoUrlat CoinGecko's free public tierinstead of the paid
pro-apihost. Neither change is in this diff — bothwere reverted before committing; they only existed long enough to prove the
real, unmodified
/v2/ratescode path end-to-end against live Binance +live (free-tier) CoinGecko data.
Result,
curl localhost:3333/v2/rates:bstock-*entries, allprovider: "coingecko", 0 entriestagged
provider: "binance".bstock-tslab:rates.usd 323.87,rates.btc 0.00504911,change24h -0.059,change7d 5.317bstock-nvdab:rates.usd 216.12,rates.btc 0.00336929,change24h 3.511,change7d 9.29bstock-sndkb:rates.usd 1411.78,rates.btc 0.02200954,change24h 6.242,change7d 27.286$327.35 vs
bstock-tslab$323.87 (~1% off); NVDA $215.61 vsbstock-nvdab$216.12 (~0.2% off); SNDK $1,435.69 vs
bstock-sndkb$1,411.78 (~1.7% off— SanDisk was mid-surge on real HBF-standard news at query time, and the
bStock's
change24h/change7dreflect the same move).cryptoarray length: 420 (364 non-bStock + 56 bStock). Non-bStock entries are real, live CoinGecko data (
bitcoin,ethereum,tether,binancecoin,usd-coin, ...) confirming bStocks are additive,not replacing existing coverage.
errors: {"cryptocompare":true, "livecoinwatch":true}reflects the missing paid keys in this sandbox, nota bug in this branch.
Test plan
npx tsc --noEmit— cleannpx jest --runInBand tests/binanceProvider.spec.ts tests/bstocks.spec.ts tests/mergeCrypto.spec.ts— 24/24 passing/v2/ratesverification against real Binance (+ free-tier CoinGecko) — see abovenpm run lint— no-op repo-wide, see point 4 aboveCo-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f