Skip to content

bStocks: Binance-sourced synthetic markets in /v2/rates - #39

Open
stultusmundi wants to merge 9 commits into
masterfrom
bstocks
Open

bStocks: Binance-sourced synthetic markets in /v2/rates#39
stultusmundi wants to merge 9 commits into
masterfrom
bstocks

Conversation

@stultusmundi

Copy link
Copy Markdown

Summary

Adds Binance bStocks (tokenized US equities on BNB Smart Chain) as synthetic
market entries in GET /v2/rates, sourced live from Binance's public Spot
API. Design spec: docs/superpowers/specs/2026-08-04-bstocks-integration-design.md
in the ZelTreZ repo (§4.2 covers this repo's contract).

  • New Binance provider client (src/services/providers/binance.ts) —
    singleton, no API key, tokenised-asset list + Spot 24h/7d tickers, with a
    last-known-good ticker cache.
  • New getBstockPrices() assembler (src/services/bstocks.ts) — intersects
    the BSC-listed tokenised-asset universe with TRADING-status Spot symbols,
    quoted in USDT (verified live — no USDC pairs exist), converts to BTC
    via the same ticker batch's BTCUSDT.
  • Wired into zelcoreRatesV2.getAll() and served through the existing
    /v2/rates route.

Emitted ids: bstock-<assetCode lowercase> (e.g. bstock-tslab), always
tagged provider: "coingecko"not "binance". The ZelCore client does
no provider-prefix parsing; it keys its market store on the literal string
${provider}-${id}, and the sibling api repo advertises each bStock's
coinInfo.coingeckoID as bstock-<code>. The two literals only meet if the
provider here is exactly "coingecko" — any other value misses silently,
with no error. This id/provider pairing is a cross-repo contract; do not
change it in isolation.

Please read before reviewing the diff

1. The positional-merge fix is the highest-impact change in this PR, and it
affects every provider, not just bStocks.
The old mergeDeep in
apiServices.ts's serviceRefresher overlaid the freshly-fetched crypto
array onto the previous one by array index. That's only correct while
every provider block returns exactly the same row count in the same order.
When a block shrank (a provider outage, a delisted coin), two things broke:
fields from the old entry at that index leaked onto a different coin (e.g.
a CryptoCompare row inheriting CoinGecko's stale rank/change7d), and
entries past the new, shorter length lived on as stale duplicates. Because
the ZelCore client re-keys on ${provider}-${id} with last-write-wins, and
those stale duplicates sat after the fresh ones in the array, wallet users
were served the stale price on any refresh cycle where a provider
block's row count shifted. Reproduced with a concrete input (simulated
CryptoCompare outage: 40 duplicate keys, client resolved to the stale price
instead of the fresh one). Fixed by replaceCryptoByKey — a key-based
rebuild from the fresh fetch alone (dedup by ${provider}-${id},
last-write-wins, no merge with the previous array). Two behaviour changes
worth knowing: repeated provider+id entries collapse to one at the first
occurrence's position, and an entry the fetch no longer produces disappears
immediately instead of persisting from the previous cycle.

2. The crypto.length > 300 degraded-response guard now counts non-bStock
rows only.
That floor exists to reject an under-strength provider response
before it overwrites good in-memory data. It was calibrated before bStocks
existed; the ~56 synthetic bStock rows would otherwise pad the count and let
a genuinely degraded CoinGecko/CryptoCompare/LiveCoinWatch response sail
past the guard, truncating /v2/rates for real assets.

3. Binance returns a halted symbol present, not absent, during a trading
halt (e.g. around a stock split) — with lastPrice: "0.00000000".
Verified
live: 20/20 requested BREAK-status symbols came back present in the ticker
response, 9/20 of those priced at zero. An absent-symbol fallback would have
been dead code for this case, and accepting the zero would both serve a $0
price and poison the last-known-good cache. Prices are therefore accepted
only when finite and strictly positive (Number.isFinite(px) && px > 0),
both for what's served and for what's written to the last-known-good store.

4. npm run lint is a no-op repo-wide, unrelated to this branch.
npx eslint ./ --format json lints exactly one file, .eslintrc.js itself,
and zero .ts files — .eslintrc.js uses parserOptions.parser: babel-eslint with no TypeScript parser, and ESLint defaults to .js only.
Every "lint clean" claim in this repo's history is vacuously true.
npx tsc --noEmit is the only real static check and is clean on this
branch. Flagging as a follow-up; not something this PR fixes.

5. Pre-existing debt this PR does not fix: mergeDeep is still
positional for ratesV2.fiat, v1 rates, and marketsUSD in
apiServices.ts — the same stale-tail class of bug described in point 1,
just not yet observed to matter for those paths (v1 rates/marketsUSD are
value-not-array shaped per currency code, so the failure mode is different in
practice; ratesV2.fiat is a small, rarely-outaged list from a single
provider). Worth its own ticket rather than folding into this change.

Live verification (real, not simulated)

No automated test proves bStock entries actually reach /v2/rates against
real Binance, so I booted the service locally and hit the live endpoint.

Environment note: this sandbox has no paid CoinGecko Pro / CryptoCompare /
LiveCoinWatch API keys, and express-prometheus-middleware's prom-client
peer dependency is missing from package.json (pre-existing, unrelated to
this branch — flagging separately, not fixed here). To get a real listening
server for this verification I temporarily (1) installed prom-client
locally and (2) pointed config.coinGeckoUrl at CoinGecko's free public tier
instead of the paid pro-api host. Neither change is in this diff — both
were reverted before committing; they only existed long enough to prove the
real, unmodified /v2/rates code path end-to-end against live Binance +
live (free-tier) CoinGecko data.

Result, curl localhost:3333/v2/rates:

  • 56 bstock-* entries, all provider: "coingecko", 0 entries
    tagged provider: "binance".
  • 3 examples:
    • bstock-tslab: rates.usd 323.87, rates.btc 0.00504911,
      change24h -0.059, change7d 5.317
    • bstock-nvdab: rates.usd 216.12, rates.btc 0.00336929,
      change24h 3.511, change7d 9.29
    • bstock-sndkb: rates.usd 1411.78, rates.btc 0.02200954,
      change24h 6.242, change7d 27.286
  • Sanity check against real share prices (fetched live, same session): TSLA
    $327.35 vs bstock-tslab $323.87 (~1% off); NVDA $215.61 vs bstock-nvdab
    $216.12 (~0.2% off); SNDK $1,435.69 vs bstock-sndkb $1,411.78 (~1.7% off
    — SanDisk was mid-surge on real HBF-standard news at query time, and the
    bStock's change24h/change7d reflect the same move).
  • Total crypto array length: 420 (364 non-bStock + 56 bStock). Non-
    bStock entries are real, live CoinGecko data (bitcoin, ethereum,
    tether, binancecoin, usd-coin, ...) confirming bStocks are additive,
    not replacing existing coverage. errors: {"cryptocompare":true, "livecoinwatch":true} reflects the missing paid keys in this sandbox, not
    a bug in this branch.

Test plan

  • npx tsc --noEmit — clean
  • npx jest --runInBand tests/binanceProvider.spec.ts tests/bstocks.spec.ts tests/mergeCrypto.spec.ts — 24/24 passing
  • Live /v2/rates verification against real Binance (+ free-tier CoinGecko) — see above
  • npm run lint — no-op repo-wide, see point 4 above

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f

stultusmundi and others added 8 commits August 5, 2026 12:21
Adds a Binance singleton provider (mirroring CoinGecko's shape) that fetches
the tokenised bStocks universe and Spot 24h/7d tickers from Binance's public
endpoints, no API key required. Exposes getTokenisedAssets (1h cache, BSC-only
filter), getTradingSymbols (1h cache, TRADING-status filter from exchangeInfo),
getTicker24h (60s cache, single batched call), and getTicker7d (60s cache,
chunked to 20 symbols/request via the existing arraySplit util).

Ticker fetches keep a last-known-good value per symbol so a halted/omitted
symbol (e.g. during a stock-split trading break) or an outright failed
refresh still serves the previous price instead of dropping the entry.

Fixtures are built from live curls of both Binance endpoints rather than
assumed shapes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Binance does not omit a halted symbol from the ticker response - it returns
the symbol present with lastPrice "0.00000000". Measured live: of 20
BREAK-status symbols requested, 20 came back present and 9 were priced at
zero. So the absent-symbol fallback was dead code for the exact scenario it
was written for, and the zero was accepted as fresh AND written to the
last-known-good store - poisoning it, so a later outage would serve $0
forever rather than the real last price.

mergeTickers now admits only finite, strictly positive prices, both to the
served set and to the store. Adds lastGoodAgeMs() so a caller can tell a live
price from one carried through a long halt, and sorts the ticker cache keys
so the same request in a different symbol order still hits cache.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Intersects the Binance tokenised-asset universe (BSC-listed, from Task 1)
with TRADING Spot symbols to emit CryptoPrice[] entries for bStocks, quoted
against BTCUSDT fetched in the same batch so both legs share one venue.
Ids are bstock-<code> under provider "coingecko" per the cross-repo id
contract with the sibling api repo. A module-level last-known-good map
keeps serving a symbol's previous price across a refresh where it's
omitted (CEX halt around a stock split), rather than dropping it.

Verified against live Binance endpoints: 66 tokenised assets are all
BSC-listed, and intersecting with exchangeInfo's TRADING Spot symbols
yields 56/66 tradable today (the other 10, e.g. NFLXB/ASMLB, aren't
listed on Spot at all yet) — matching the brief's expected ~56/66.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
… path

The guard against a missing or zero BTCUSDT was correct but untested, so a
later simplification could silently put Infinity or NaN into rates.btc with
nothing in CI to catch it - the same shape of bug that reached a signing path
in the sibling api repo. Mutation-tested: dropping the btcUsd half of the
guard fails both new tests.

Also pins that one halted asset does not affect its siblings in the same
batch, and that the feature flag actually suppresses output.

Corrects two JSDoc claims that came from the plan text: the client does no
provider-prefix parsing (applyMarkets and use-fiat.js each build the literal
`${provider}-${id}` and the strings simply have to match), and Binance does
not omit halted symbols - it returns them present at zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Adds a fifth try/catch provider block to zelcoreRatesV2.getAll() that
appends getBstockPrices() (Task 2) into the crypto array, wrapped so a
Binance/bStocks outage sets errors.binance = true instead of throwing out
of getAll(). Also excludes synthetic `bstock-*` ids from the CoinGecko
harvest in coinAggregatorIDs.ts so rates-api never queries CoinGecko for
them.

Fixes a pre-existing positional-merge bug in apiServices.ts: the crypto
refresh merge used mergeDeep, which walks target/source arrays by index.
Since `processed` in zelcoreRatesV2.getAll() is a concatenation of four
independent try/catch provider blocks, its length and per-index identity
shift between refresh cycles whenever any block throws or an upstream API
returns a different row count -- both routine occurrences. Concretely: if
CoinGecko's block (rank/change7d present) fails one cycle while
CryptoCompare's block (no rank/change7d) succeeds, mergeDeep deep-merges
the old CoinGecko entry at index 0 with the new CryptoCompare entry at
index 0 -- id/provider/rates get correctly overwritten, but `rank` survives
from the stale CoinGecko entry, producing a CryptoCompare coin wearing a
foreign coin's rank. Separately, if the new array is shorter than the old
one, mergeDeep's source.forEach never visits the trailing old indices, so
entries missing from the new fetch persist in the output forever with
frozen data instead of being dropped.

Replaces the crypto merge with mergeCryptoByKey, which discards the stale
target and rebuilds the array from source only, keyed by
`${provider}-${id}`. tests/mergeCrypto.spec.ts pins this: one test
documents mergeDeep's corruption (still used unchanged for fiat/rates/
marketsUSD) on realistic shape-mismatched input, one confirms
mergeCryptoByKey fixes it, and one confirms mergeCryptoByKey produces
identical output to mergeDeep for a normal, well-ordered refresh so
existing consumers see no behavioural change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
…merge key

The crypto.length > 300 sanity floor was calibrated before bStocks existed.
With ~56 synthetic entries added, a degraded CoinGecko response (250 rows
instead of ~342, which it returns silently rather than throwing) now clears
the floor where it used to be blocked: 250+39+2 = 291 was rejected,
250+39+2+56 = 347 is accepted. Combined with the wholesale replace, that
truncates /v2/rates and drops ~100 coins for the cycle. The floor now counts
real-provider rows only.

mergeCryptoByKey is renamed replaceCryptoByKey and its unused _target
parameter dropped. It never merged - it discards target entirely - and the
old signature invited a maintainer to wire target back in, which would throw
on the first refresh cycle where ratesV2.crypto is genuinely undefined.

A mutation run showed a bare spread of source passed every test in the file:
nothing pinned that provider is part of the key, that duplicates collapse, or
that the last write wins. Two tests now cover those; all three surviving
mutants are killed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Documents the synthetic Binance bStock entries in /v2/rates: universe
selection (BSC contract + TRADING Spot symbol), USDT quoting, the
provider:"coingecko" id contract with the client and the sibling api
repo, and the halted-symbol/last-known-good behavior. Regenerates the
committed docs/ typedoc tree to pick up the new bstocks/binance
modules and types.

Bumped the typedoc devDependency ^0.26.7 -> ^0.28.0: typedoc-plugin-
markdown@4.12 (already the installed version, satisfying the existing
^4.2.7 range) requires typedoc 0.28.x as a peer, so `npx typedoc` -
the docs-refresh step this same README documents - was silently
broken on a clean install before this change. Dev-only, no runtime
effect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
…ry-forward, outage visibility, refresh bounding

Addresses the whole-branch review of bstocks (PR #39):

1. binance.ts: lastGoodTicker was keyed on bare symbol, so getTicker24h
   and getTicker7d shared one fallback slot. Whichever window last
   wrote silently overwrote the other's priceChangePercent/quoteVolume,
   so a 24h fallback could serve the 7-day change (and ~7x-inflated
   volume) as the 24-hour figure. Now keyed per `${window}:${symbol}`;
   mergeTickers and lastGoodAgeMs take an explicit window.

2. apiServices.ts: serviceRefresher rebuilt ratesV2.crypto from the
   fresh fetch alone once the >300-row floor passed. A provider whose
   block failed this cycle (CryptoCompare's ~39 rows, LiveCoinWatch's
   2) would vanish from /v2/rates outright whenever the remaining
   providers alone still cleared the floor -- new in this branch, since
   replaceCryptoByKey has no positional stale tail to fall back on.
   Now carries forward only the rows belonging to providers present in
   ratesV2Fetched.errors, then key-merges with fresh data last so fresh
   always wins.

3. bstocks.ts / zelcoreRatesV2.ts: every failure inside getBstockPrices
   is caught internally, so a total Binance outage looked identical to
   a healthy refresh while quietly re-serving frozen prices forever.
   Added isBstocksDegraded() (true when a refresh prices zero symbols
   fresh while last-known-good is non-empty), wired into errors.binance,
   and bounded last-known-good to config.bstocksLastGoodMaxAgeMs (7
   days, chosen because the outage this exists for -- a stock split
   halt -- is naturally multi-day).

4. zelcoreRatesV2.ts / binance.ts: a Binance outage cost up to ~92s
   (AxiosWrapper's retry budget across getTokenisedAssets,
   getTradingSymbols, and both ticker windows), stalling the refresh
   of all other providers behind it every 30s cycle. getBstockPrices()
   is now raced against a 10s timeout, and getTokenisedAssets/
   getTradingSymbols negatively-cache a failure for
   config.binanceFailureCacheMs (60s) so the retry storm doesn't repeat
   every cycle. The timeout's setTimeout is .unref()'d since
   Promise.race never cancels the losing branch.

5. config/index.ts: bStocksEnabled now reads
   process.env.BSTOCKS_ENABLED !== 'false' instead of a literal, so it
   can be disabled without a redeploy.

6. bstocks.ts: rank is no longer hardcoded to 0 (which sorted every
   bStock ahead of Bitcoin in ascending rank order) -- omitted, matching
   the convention CryptoCompare's rows already use elsewhere in this
   repo.

7. New test coverage: serviceRefresher (previously zero coverage) for
   the small-provider-outage guard/carry-forward interaction; a
   total-outage integration case pinning that getBstockPrices never
   rejects; and a multi-asset property assertion over the
   coingecko/bstock-<code> cross-repo id contract.

Each fix has a test that was verified to fail before the change and
pass after (confirmed via targeted git-stash reverts per file).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
@stultusmundi

Copy link
Copy Markdown
Author

Fixes from final whole-branch review

This commit (87b3888) addresses all 6 findings from the final review, plus the 3 test-coverage blind spots called out alongside them.

1. 24h/7d last-known-good collision (binance.ts)lastGoodTicker was keyed on bare symbol, so getTicker24h/getTicker7d shared one fallback slot; whichever window last wrote silently overwrote the other's priceChangePercent/quoteVolume. Now keyed per ${window}:${symbol}; mergeTickers/lastGoodAgeMs take an explicit '24h' | '7d' window (omitting it on lastGoodAgeMs reports the freshest of the two).

2. Coins vanishing on a small-provider outage (apiServices.ts)serviceRefresher rebuilt ratesV2.crypto from the fresh fetch alone once the >300-row floor passed, so a CryptoCompare (~39 rows) or LiveCoinWatch (2 rows) outage dropped those coins from /v2/rates entirely once the remaining providers alone still cleared 300. Fixed exactly as prescribed: carry forward only rows belonging to providers present in ratesV2Fetched.errors, then key-merge with fresh data last so fresh always wins.

3. Invisible total Binance outage + unbounded staleness (bstocks.ts / zelcoreRatesV2.ts) — every failure inside getBstockPrices is caught internally, so a total outage looked identical to a healthy refresh while quietly re-serving frozen prices forever. Added isBstocksDegraded() (true when a refresh prices zero symbols fresh while last-known-good is non-empty), wired into errors.binance, and bounded last-known-good to a new config.bstocksLastGoodMaxAgeMs (7 days — chosen because the outage this exists for, a stock-split halt, is naturally multi-day).

4. Binance outage slowing the whole refresh cycle (zelcoreRatesV2.ts / binance.ts)getBstockPrices() is now raced against a 10s timeout at the call site, and getTokenisedAssets/getTradingSymbols negatively-cache a failure into longCache for config.binanceFailureCacheMs (60s) so the AxiosWrapper retry storm doesn't repeat every 30s cycle. Note: Promise.race never cancels the losing branch, so the timeout's setTimeout is .unref()'d — otherwise it holds the process open for the tail of the 10s on every healthy cycle (caught this via a lingering-timer test hang during verification).

5. No env kill switch (config/index.ts)bStocksEnabled now reads process.env.BSTOCKS_ENABLED !== 'false' instead of a literal.

6. rank: 0 sorting bStocks ahead of Bitcoin (bstocks.ts)rank is now omitted entirely (matching CryptoCompare's rows elsewhere in this repo) instead of hardcoded to 0.

Test blind spots closed: serviceRefresher (previously zero coverage) now has tests for the guard/carry-forward interaction; a total-outage case pins that getBstockPrices never rejects and correctly reports degraded; a multi-asset property test asserts the coingecko/bstock-<code> cross-repo id contract across 5 assets instead of 1.

Verification: npx tsc --noEmit is clean. Full suite (binanceProvider, bstocks, mergeCrypto, plus new config, serviceRefresher, zelcoreRatesV2 specs) is 37/37 passing. Each finding's test was confirmed to fail against the pre-fix code (via targeted git stash of just that file) and pass after — including two findings (3 and 6) that share a file, isolated via a surgical single-line revert to avoid conflating them. productionCompare*.spec.ts still fail for the pre-existing, unrelated reason (no local server on :3333).

🤖 Generated with Claude Code

…hing

The 10s bound added for the slow-refresh finding resolved to [] on timeout,
which reintroduced the vanishing-rows failure it was meant to prevent, for
bStocks specifically. The provider carry-forward in apiServices keys on the
row's provider, but bStock rows carry provider "coingecko" while their failure
is reported under errors.binance - so nothing could ever carry them. On a hung
Binance the first cycle produced zero bStock rows AND an empty errors object:
the wallet shows $0 with no banner. Since AxiosWrapper's worst case is ~23s
against a 10s bound, the very first cycle of any outage hits it, and rows then
blink out roughly every third cycle as the 60s negative cache expires.

The timeout branch now serves the pruned last-known-good snapshot and flags
the cycle degraded, since freshPricedLastRun still reflects an earlier run
when the race is lost.

isBstocksDegraded no longer requires lastGood to be non-empty: a cold start
during an outage had nothing cached, so it reported a healthy service that was
serving no bStocks at all. It now returns false when the feature is disabled,
so "off" stays distinguishable from "broken".

Also makes BSTOCKS_ENABLED case-insensitive - BSTOCKS_ENABLED=FALSE silently
left the feature on - and documents it in .env.example, where an operator can
actually find it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnpQSycV6gfbF9JqRf4x1f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant