Skip to content

[DO NOT MERGE] refactor(sdk): unify CRDT codec into a parse-once pipeline - #1522

Draft
cyaiox wants to merge 1 commit into
stack/04-validatorfrom
stack/05-codec
Draft

[DO NOT MERGE] refactor(sdk): unify CRDT codec into a parse-once pipeline#1522
cyaiox wants to merge 1 commit into
stack/04-validatorfrom
stack/05-codec

Conversation

@cyaiox

@cyaiox cyaiox commented Aug 5, 2026

Copy link
Copy Markdown
Member

Phase 3 of the authoritative-server network refactor — the highest-
risk phase, executed with byte-identical wire goldens (the 7
characterization snapshots pass unmodified):

  • network/codec.ts (server/utils.ts renamed, chunking.ts absorbed) is
    the sole owner of read/translate/pack; nothing else in the layer
    parses or packs CRDT bytes
  • server ingest walks each inbound buffer exactly once: validation
    consumes the parsed form and broadcast re-packs the retained byte
    slices — the structural re-parse inside the old chunker is gone
    (2 parses → 1 per buffer, asserted by a spy-based test)
  • packChunks: one place chunk boundaries are decided, O(1) size
    accounting via currentWriteOffset, lazy pull with copy-before-reuse
    so producers can yield views over a scratch buffer instead of
    allocating 10KB per message
  • one oversize error path shared by dumps and broadcasts, always-on,
    naming component, entity, byte size, and the 12KB ceiling (chore: change s3 secrets and comment an installable package to test #8 green)
  • state.ts dump path now measures the actual emitted slice instead of
    payload-only bytes (was undercounting headers by ~28B/message)

Red defect test #8 flips to passing; #10, #11, #12, #13 verified still
red. chunkCrdtMessages survives only as the characterization pin's
composition (ponytail marker with removal path).

Claude-Session: https://claude.ai/code/session_01Kzo6zwrn1CA79S4RN1dgsU

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kzo6zwrn1CA79S4RN1dgsU


📚 Stack (splits #1505 — merge top-down within each stack)

Network layer (each PR is one self-contained, independently-green commit):

  1. test(sdk): characterization baseline for the network layer refactor #1518 test harness — characterization baseline + red defect suite (this stack's root)
  2. [DO NOT MERGE] refactor(sdk): network layer foundations — break cycles, single isServer, logging discipline #1519 foundations — cycle break, single isServer, logging discipline
  3. [DO NOT MERGE] feat(sdk): hydration FSM — deterministic late-join and reconnect convergence #1520 hydration FSM — late-join/reconnect correctness
  4. [DO NOT MERGE] feat(sdk): default-deny server validation, fixed correction path, O(1) entity lookup #1521 validator — default-deny, correction path, entity index
  5. [DO NOT MERGE] refactor(sdk): unify CRDT codec into a parse-once pipeline #1522 codec — parse-once pipeline (wire goldens byte-identical)
  6. [DO NOT MERGE] feat(sdk): clients target the authoritative server; registry collisions; players dedupe #1523 topology — targeted sends, registry collisions, players dedupe
  7. [DO NOT MERGE] refactor(sdk): parameterize network boot, document the host contract #1524 boot + host-conformance doc
  8. [DO NOT MERGE] refactor(sdk): ponytail sweep — delete what the refactor no longer needs #1525 simplification sweep (−95 lines)

sdk-commands (independent of the stack above):

After a parent squash-merges, the child needs git rebase --onto origin/auth-server <old-parent> <child> — ping the session and it gets restacked.

Phase 3 of the authoritative-server network refactor — the highest-
risk phase, executed with byte-identical wire goldens (the 7
characterization snapshots pass unmodified):

- network/codec.ts (server/utils.ts renamed, chunking.ts absorbed) is
  the sole owner of read/translate/pack; nothing else in the layer
  parses or packs CRDT bytes
- server ingest walks each inbound buffer exactly once: validation
  consumes the parsed form and broadcast re-packs the retained byte
  slices — the structural re-parse inside the old chunker is gone
  (2 parses → 1 per buffer, asserted by a spy-based test)
- packChunks: one place chunk boundaries are decided, O(1) size
  accounting via currentWriteOffset, lazy pull with copy-before-reuse
  so producers can yield views over a scratch buffer instead of
  allocating 10KB per message
- one oversize error path shared by dumps and broadcasts, always-on,
  naming component, entity, byte size, and the 12KB ceiling (#8 green)
- state.ts dump path now measures the actual emitted slice instead of
  payload-only bytes (was undercounting headers by ~28B/message)

Red defect test #8 flips to passing; #10, #11, #12, #13 verified still
red. chunkCrdtMessages survives only as the characterization pin's
composition (ponytail marker with removal path).

Claude-Session: https://claude.ai/code/session_01Kzo6zwrn1CA79S4RN1dgsU
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Test this pull request

  • The @dcl/sdk package can be tested in scenes by running

    npm install "https://sdk-team-cdn.decentraland.org/@dcl/js-sdk-toolchain/branch/stack/05-codec/dcl-sdk-7.25.1-31013698504.commit-9437977.tgz"
  • The @dcl/js-runtime package can be tested in scenes by running

    npm install "https://sdk-team-cdn.decentraland.org/@dcl/js-sdk-toolchain/branch/stack/05-codec/@dcl/js-runtime/dcl-js-runtime-7.25.1-31013698504.commit-9437977.tgz"
  • To test with npx init

    export SDK_COMMANDS="https://sdk-team-cdn.decentraland.org/@dcl/js-sdk-toolchain/branch/stack/05-codec/dcl-sdk-commands-7.25.1-31013698504.commit-9437977.tgz"
    npx $SDK_COMMANDS init
  • The /changerealm command to test test in-world

    /changerealm https://sdk-team-cdn.decentraland.org/ipfs/stack/05-codec-e2e
    
  • You can preview this build entering:
    https://playground.decentraland.org/?sdk-branch=stack/05-codec

@cyaiox cyaiox changed the title refactor(sdk): unify CRDT codec into a parse-once pipeline [DO NOT MERGE] refactor(sdk): unify CRDT codec into a parse-once pipeline Aug 5, 2026
@cyaiox
cyaiox marked this pull request as draft August 5, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant