[RFC] Wire contract + mechanical wire-compat CI gate (complement to #1920)

docs/rfc/wire-compat/ci/wire-compat-gate.yml

@@ -0,0 +1,64 @@
+# Wire-compatibility CI gate for Mooncake Store.
+#
+# Complement to issue #1920 (rolling-upgrade RFC). #1920 proposes "enforce via
+# code review"; this job enforces mechanically. It rebuilds the wire-contract
+# golden generator from the PR's tree, self-checks the codec against recorded
+# reference codes, then diffs the live wire contract (coro_rpc function-ids +
+# struct_pack argument type-codes for every WrappedMasterService handler) against
+# the checked-in golden. Any drift -- a renamed RPC, a reordered/added bare
+# argument, or a layout change to a serialized struct -- fails the PR.
+#
+# An intentional, reviewed major-version wire break is landed by bumping the
+# protocol version and committing a regenerated golden (./check_wire_contract.sh
+# --update), which makes the break an explicit, reviewable diff.
+
+name: wire-compat-gate
+
+on:
+  pull_request:
+    paths:
+      - "mooncake-store/src/rpc_service.cpp"
+      - "mooncake-store/include/rpc_service.h"
+      - "mooncake-store/include/replica.h"
+      - "mooncake-store/include/types.h"
+      - "mooncake-store/include/allocator.h"
+      - "mooncake-store/src/master_client.cpp"
+      - "docs/rust-rewrite/wire-compat-rfc/gate/**"
+  workflow_dispatch: {}
+
+jobs:
+  wire-contract:
+    runs-on: ubuntu-22.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install toolchain
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends g++ git ca-certificates
+          g++ --version | head -1
+
+      - name: Fetch yalantinglibs headers
+        run: |
+          # The standalone gate needs only struct_pack + util/function_name.h.
+          # Pin the same yalantinglibs revision the build uses; header-only checkout.
+          git clone --depth 1 https://github.com/alibaba/yalantinglibs.git /tmp/ylt
+          echo "YLT_INCLUDE=/tmp/ylt/include" >> "$GITHUB_ENV"
+
+      - name: Run wire-contract gate (standalone golden)
+        run: |
+          chmod +x docs/rust-rewrite/wire-compat-rfc/gate/check_wire_contract.sh
+          docs/rust-rewrite/wire-compat-rfc/gate/check_wire_contract.sh \
+            --ylt "${YLT_INCLUDE}"
+
+      # Optional second face: the in-build ctest computes the same codes over the
+      # real production structs (Segment, *Request/*Response) and carries the
+      # cross-version interop round-trips. Enable once the test target is wired
+      # into mooncake-store/tests/CMakeLists.txt.
+      #
+      # - name: Build + run in-build wire-contract ctest
+      #   run: |
+      #     cmake -S . -B build -DBUILD_TESTING=ON
+      #     cmake --build build --target wire_contract_test
+      #     ctest --test-dir build -R wire_contract --output-on-failure

docs/rfc/wire-compat/gate/README.md

@@ -0,0 +1,60 @@
+# Wire-contract gate
+
+Mechanical pre-merge gate for the Mooncake Store master coro_rpc surface. See
+`../WIRE-COMPAT-RFC.md` for the full proposal (a complement to issue #1920).
+
+## Files
+
+- `wire_contract_gen.cpp` -- standalone generator. Builds with only the
+  yalantinglibs headers. Emits, per handler, the real coro_rpc `function_id` and
+  the real struct_pack argument `arg_type_code`. Has a `--selfcheck` that proves
+  the codec engine against the recorded #2288 reference codes.
+- `wire_contract_golden.txt` -- the checked-in golden table (59 handlers).
+- `check_wire_contract.sh` -- the CI driver: rebuild, self-check, regenerate,
+  diff against the golden. Exit 0 = intact, 1 = drift, 2 = build error.
+- `wire_contract_test.cpp` -- in-build ctest face. Includes the real Mooncake
+  headers, pins codes over the actual production structs, and carries the #2551
+  cross-version interop round-trips.
+
+## Run
+
+```
+# needs a yalantinglibs install (struct_pack + util/function_name.h only)
+./check_wire_contract.sh --ylt /path/to/ylt-install/include
+./wire_contract_gen --selfcheck       # 4/4 reference codes OK
+```
+
+Self-check expected output:
+
+```
+[selfcheck] ExistKey bare<string>                              got=0x9dcffa76 want=0x9dcffa76 OK
+[selfcheck] PutStart #2288-era 4-arg (3-field RC)              got=0xfad0c534 want=0xfad0c534 OK
+[selfcheck] PutStart + bare trailing string (#2288 break)      got=0x22f8edba want=0x22f8edba OK
+[selfcheck] PutStart + compatible<string> (the fix)            got=0xfad0c534 want=0xfad0c534 OK
+```
+
+Clean tree: `[gate] PASS: master wire contract unchanged (59 handlers)`.
+
+## Demonstrating drift (for reviewers)
+
+Append a bare trailing argument to any handler stub plus its type-code row, e.g.
+add `const std::string&` to `PutEnd`, rebuild and re-run: the gate prints a diff
+and exits 1. The `function_id` is unchanged (name unchanged); the `arg_type_code`
+shifts -- exactly the silent, compiles-cleanly break class #2288 was. Revert to
+return to green.
+
+## Updating the golden (intentional wire bump only)
+
+```
+./check_wire_contract.sh --update    # regenerate; bump the major protocol version too
+```
+
+This makes every wire break an explicit, reviewable commit.
+
+## Notes
+
+- The compiled `wire_contract_gen` binary is a build artifact; do not commit it.
+- Generated/temp files (`.wire_contract_live.txt`, `.build.log`,
+  `.wire_contract.diff`) are not committed.
+- Toolchain used for the demonstration: g++ 11.4 (C++20), yalantinglibs as
+  vendored by Mooncake v0.3.11.post1 (commit e9c6107).

docs/rfc/wire-compat/gate/VERIFY-transcript.txt

@@ -0,0 +1,17 @@
+### Build
+build: OK
+
+### Self-check (codec faithfulness)
+[selfcheck] ExistKey bare<string>                              got=0x9dcffa76 want=0x9dcffa76 OK
+[selfcheck] PutStart #2288-era 4-arg (3-field RC)              got=0xfad0c534 want=0xfad0c534 OK
+[selfcheck] PutStart + bare trailing string (#2288 break)      got=0x22f8edba want=0x22f8edba OK
+[selfcheck] PutStart + compatible<string> (the fix)            got=0xfad0c534 want=0xfad0c534 OK
+selfcheck rc=0
+
+### Gate on clean tree
+[selfcheck] PutStart + bare trailing string (#2288 break)      got=0x22f8edba want=0x22f8edba OK
+[selfcheck] PutStart + compatible<string> (the fix)            got=0xfad0c534 want=0xfad0c534 OK
+[gate] PASS: master wire contract unchanged (59 handlers)
+
+### Golden handler count
+handlers=59

docs/rfc/wire-compat/gate/check_wire_contract.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# check_wire_contract.sh -- the CI gate driver.
+#
+# Rebuilds the wire-contract generator from the CURRENT source tree, emits the
+# live wire-contract table, and diffs it against the checked-in golden. Any drift
+# in a function-id or an arg-type-code (i.e. any change that would break an
+# existing-version peer) fails the build with a readable diff.
+#
+# Usage:
+#   check_wire_contract.sh [--ylt <ylt-install-include-dir>] [--update]
+#
+#   --update   regenerate the golden instead of checking (use only for an
+#              intentional, reviewed major-version wire bump).
+#
+# Exit codes: 0 = contract intact; 1 = drift detected; 2 = build/setup error.
+set -uo pipefail
+
+here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ylt="${YLT_INCLUDE:-/tmp/compat-official/ylt-install/include}"
+update=0
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --ylt) ylt="$2"; shift 2;;
+    --update) update=1; shift;;
+    *) echo "unknown arg: $1" >&2; exit 2;;
+  esac
+done
+
+gen="$here/wire_contract_gen"
+src="$here/wire_contract_gen.cpp"
+golden="$here/wire_contract_golden.txt"
+
+if [ ! -f "$ylt/ylt/struct_pack.hpp" ]; then
+  echo "ERROR: yalantinglibs headers not found at $ylt (set YLT_INCLUDE or --ylt)" >&2
+  exit 2
+fi
+
+echo "[gate] building generator from $src"
+if ! g++ -std=c++20 -I"$ylt" "$src" -o "$gen" 2> "$here/.build.log"; then
+  echo "ERROR: generator failed to build:" >&2
+  cat "$here/.build.log" >&2
+  exit 2
+fi
+
+echo "[gate] self-check (codec faithfulness vs recorded #2288 reference codes)"
+if ! "$gen" --selfcheck; then
+  echo "ERROR: codec self-check failed -- the gate cannot trust its own type codes" >&2
+  exit 2
+fi
+
+live="$here/.wire_contract_live.txt"
+"$gen" > "$live"
+
+if [ "$update" = "1" ]; then
+  cp "$live" "$golden"
+  echo "[gate] golden UPDATED -> $golden (intentional wire bump; ensure major version was bumped)"
+  exit 0
+fi
+
+if [ ! -f "$golden" ]; then
+  echo "ERROR: golden $golden missing; run with --update to create it" >&2
+  exit 2
+fi
+
+if diff -u "$golden" "$live" > "$here/.wire_contract.diff"; then
+  echo "[gate] PASS: master wire contract unchanged ($(grep -vc '^#' "$golden") handlers)"
+  rm -f "$live"
+  exit 0
+else
+  echo "" >&2
+  echo "============================================================" >&2
+  echo " WIRE CONTRACT DRIFT DETECTED -- this change alters the" >&2
+  echo " on-wire RPC identity and will break existing-version peers." >&2
+  echo "============================================================" >&2
+  cat "$here/.wire_contract.diff" >&2
+  echo "" >&2
+  echo "If this is an intentional, reviewed MAJOR-version wire bump," >&2
+  echo "bump MOONCAKE_PROTOCOL_VERSION and re-run with --update." >&2
+  echo "Otherwise: revert the signature/struct change, or carry the new" >&2
+  echo "field as struct_pack::compatible<T> (additive, type-code stable)." >&2
+  rm -f "$live"
+  exit 1
+fi