Skip to content

Modernise dep stack (tonic 0.14, prost 0.14, …) + encoding-size setter - #9

Merged
tobiemh merged 6 commits into
mainfrom
tobiemh/modernize-deps
May 26, 2026
Merged

Modernise dep stack (tonic 0.14, prost 0.14, …) + encoding-size setter#9
tobiemh merged 6 commits into
mainfrom
tobiemh/modernize-deps

Conversation

@tobiemh

@tobiemh tobiemh commented May 25, 2026

Copy link
Copy Markdown
Member

What

Modernises the dep stack of the SurrealDB tikv-client fork to bring it in line with the rest of the SurrealDB workspace, and adds a Config::with_grpc_max_encoding_message_size setter so consumers can lift the 4 MB request-side cap.

Why

When SurrealDB's TiKV operational hardening PR went up against surrealdb/surrealdb-private#246, cargo deny flagged seven RUSTSEC advisories that all chained back to this crate's pinned deps:

  • rustls-webpki 0.101.x (RUSTSEC-2026-0098, -0099, -0104) and rustls-pemfile (RUSTSEC-2025-0134) — pinned via tonic 0.10rustls 0.21.
  • ansi_term (RUSTSEC-2021-0139) and atty (RUSTSEC-2024-0375) — transitive via the clap 2 dev-dep.
  • instant (RUSTSEC-2024-0384) — transitive via serial_test 0.5 / parking_lot 0.11.

Bumping the runtime deps to current versions and dropping the live-cluster integration-test dev-dep block clears all seven. The SurrealDB workspace can then drop the corresponding deny.toml ignores once it points at `0.5.0`.

The encoding-side gRPC setter is the only API addition. SurrealDB already plumbs `SURREAL_TIKV_GRPC_MAX_ENCODING_MESSAGE_SIZE` end-to-end and reads it as a no-op pending this release.

Changes

Runtime deps:

Crate Before After Note
`tonic` 0.10 0.14.5 `tls-native-roots`; clears 4× rustls* advisories
`prost` 0.12 0.14 Must match `tonic-prost 0.14`
`thiserror` 1 2
`async-recursion` 0.3 1.1
`derive-new` 0.5 0.7
`fail` 0.4 0.5
`prometheus` 0.13 0.14
`rand` 0.8 0.9 `thread_rng` → `rng`, `gen[_range]` → `random[_range]`
`lazy_static` 1 dropped Two sites moved to `std::sync::LazyLock`
`serde_json` runtime 1 dropped Only used by removed dev tests

Toolchain: `rust-toolchain.toml` bumped from `1.84.1` → `1.95.0` to satisfy the `edition2024` requirement introduced by `tonic 0.14` / `tokio 1.x`.

Proto regeneration (`proto-build/`):

  • Switched from `tonic-build 0.10` to `tonic-prost-build 0.14`.
  • Adjusted `compile_protos` call to the new `&[PathBuf]` include signature.
  • Regenerated every `src/generated/*.rs` with the new codegen output (`tonic::codec::ProstCodec` → `tonic_prost::ProstCodec`).

New API:
```rust
let config = Config::default()
.with_grpc_max_decoding_message_size(64 * 1024 * 1024)
.with_grpc_max_encoding_message_size(64 * 1024 * 1024);
```
Threaded through `TikvConnect` and `PdRpcClient::connect`, applied via `.max_encoding_message_size(…)` on the `TikvClient` channel alongside the existing decoding limit.

Dev-deps + dead artefacts dropped:

  • `[dev-dependencies]` trimmed to the minimum the live-cluster integration suite still needs (`env_logger`, `fail/failpoints`, `reqwest` with rustls, `serde_json`, `serial_test 3`, `tokio`); `clap 2`, `proptest`, `proptest-derive`, `rstest`, `tempfile` are gone.
  • Deleted `examples/` and `src/proptests/` and the matching `[[example]]` entry. `tests/` is kept and gated on the existing `integration-tests` feature; it's the only entry point that exercises a real PD+TiKV cluster.
  • Stripped the in-crate `#[cfg(test)]` modules that depended on the dropped dev-deps (proptest-derive markers on `Key`/`BoundRange`/`KvPair`; rstest cases in `src/raw/requests.rs`, `src/transaction/lock.rs`, `src/transaction/transaction.rs`; the tempfile-based test in `src/common/security.rs`).
  • These upstream artefacts aren't run from any SurrealDB workflow and are recoverable from `tikv/client-rust` history if needed.

Crate metadata: version `0.3.0-surreal.4` → `0.5.0`. Repository URL and description updated to mark this as the SurrealDB fork.

Note on the version choice: this is the first release from the SurrealDB-owned fork (default branch moved to `main`; we no longer track upstream `tikv/client-rust` versions). The crate is published as `surrealdb-tikv-client`, which already disambiguates from upstream's `tikv-client`, so the `-surreal.N` suffix is dropped in favour of plain semver. `0.5.0` also sorts unambiguously above upstream's `0.4.0` publish (2026-02-07), which kept the same vulnerable dep stack as 0.3.0.

Verification

  • `cargo run -p tikv-client-proto-build` regenerates `src/generated/*.rs` cleanly.
  • `cargo check` succeeds with 24 inherited upstream warnings, zero errors.

Follow-up for consumers

Once this is released:

  1. Bump the workspace dep in `surrealdb/surrealdb-private` to `surrealdb-tikv-client 0.5.0`.
  2. Drop the seven now-unused `RUSTSEC-*` ignores from `deny.toml`.
  3. Wire the `with_grpc_max_encoding_message_size` setter in the TiKV `Datastore::new` path so the `SURREAL_TIKV_GRPC_MAX_ENCODING_MESSAGE_SIZE` env var stops being a no-op.

tobiemh added 5 commits May 25, 2026 16:17
…etter

Brings the SurrealDB fork into line with the rest of the SurrealDB
workspace and clears seven RUSTSEC advisories that the old upstream
pins were forcing the consumer (SurrealDB) to ignore. Also adds
`Config::with_grpc_max_encoding_message_size` so consumers can lift
the 4 MB request-side cap, mirroring the existing
`with_grpc_max_decoding_message_size` setter.

Runtime deps:
- tonic 0.10 -> 0.14.5 (`tls-native-roots` feature). Clears
  RUSTSEC-2025-0134 (rustls-pemfile), RUSTSEC-2026-0098 / -0099 / -0104
  (rustls-webpki 0.101.x) pinned via the old tonic 0.10 -> rustls 0.21
  chain.
- prost 0.12 -> 0.14 (must match `tonic-prost 0.14`).
- thiserror 1 -> 2.
- async-recursion 0.3 -> 1.1, derive-new 0.5 -> 0.7, fail 0.4 -> 0.5,
  prometheus 0.13 -> 0.14.
- rand 0.8 -> 0.9 (`thread_rng()` -> `rand::rng()`,
  `Rng::gen_range` -> `Rng::random_range`, `Rng::gen` -> `Rng::random`).
- lazy_static dropped; two sites migrated to `std::sync::LazyLock`.
- `serde_json` dropped from runtime deps (unused outside the removed
  dev tests).

`rust-toolchain.toml`: bumped from 1.84.1 to 1.95.0 to satisfy the
edition2024 requirement introduced by tonic 0.14 / tokio 1.x.

Proto regeneration (`proto-build/`):
- Switched from `tonic-build 0.10` to `tonic-prost-build 0.14`.
- Adjusted `compile_protos` call to the new include-path signature
  (`&[PathBuf]`).
- Regenerated every `src/generated/*.rs` with the new codegen output.
  References to `tonic::codec::ProstCodec` are now `tonic_prost::ProstCodec`.

New API:
- `Config::grpc_max_encoding_message_size` field +
  `Config::with_grpc_max_encoding_message_size(size)` builder.
- Threaded through `TikvConnect` and `PdRpcClient::connect`, applied
  via `.max_encoding_message_size(...)` on the TikvClient channel
  alongside the existing decoding limit.

Dev-deps + dead artefacts dropped:
- `[dev-dependencies]` reduced to just `fail + tokio` for the
  remaining failpoint plumbing. The rest (clap 2, env_logger,
  serial_test 0.5, proptest, proptest-derive, reqwest, rstest,
  tempfile, serde_json) supported upstream's CLI examples and live-
  cluster integration tests; they pulled in unmaintained transitive
  crates (ansi_term -> RUSTSEC-2021-0139, atty -> RUSTSEC-2024-0375,
  instant -> RUSTSEC-2024-0384).
- Deleted `tests/`, `examples/`, `src/proptests/`, and the matching
  `[[test]]` / `[[example]]` entries in `Cargo.toml`.
- Stripped the in-crate `#[cfg(test)]` modules that depended on the
  dropped dev-deps: proptest-derive `Arbitrary` markers on `Key`,
  `BoundRange`, `KvPair`; rstest-driven cases in `src/raw/requests.rs`,
  `src/transaction/lock.rs`, `src/transaction/transaction.rs`; the
  tempfile-based test in `src/common/security.rs`. The upstream code
  is recoverable from `tikv/client-rust` git history if needed.

Crate metadata:
- Version: `0.3.0-surreal.4` -> `0.4.0-surreal.1`.
- Repository URL updated to `https://github.com/surrealdb/tikv-client`.

Verification: `cargo run -p tikv-client-proto-build` regenerates clean.
`cargo check` succeeds (24 inherited upstream warnings, no errors).
The `surrealdb/tikv-client` org policy now requires all GitHub Actions
to be referenced by full-length commit SHA rather than tag. The
existing workflow used `@v4`, `@v1`, `@v2`, and `@nextest` tags, which
the org-level allow-list rejects on every PR run.

Each `uses:` now points at the matching SHA with a trailing
`# v<X>` comment that documents the human-readable version the
SHA corresponds to, matching the GitHub-recommended pinning format.

No behavioural change; this just unblocks CI on PR #9 and any
subsequent PR.
`make check` runs with `RUSTFLAGS=-Dwarnings` and `-D clippy::all` on
top of the regenerated proto bindings and the bumped runtime deps. The
combination promoted previously-benign warnings to hard errors. Address
them so the upstream Makefile passes again under the new dep stack:

- `src/proto.rs`: add `dead_code` to the existing inner `allow` attr.
  The regenerated bindings define many message types
  (`Participant`, `StoreRecoverState`, `RegionSequenceNumberRelation`,
  …) that the client code never references — they exist only so the
  proto schema round-trips. The allow stays scoped to the generated
  tree.
- `src/transaction/transaction.rs`: `let mut new_len` inside the
  `before-commit-secondary` failpoint is only mutated when the
  `fail/failpoints` feature is active. Add a targeted
  `#[allow(unused_mut)]` so the default `--features integration-tests`
  build (without `failpoints`) doesn't error.
- `cargo clippy --fix` resolved a batch of `clone_on_copy` warnings on
  `Timestamp` (prost 0.14 derives `Copy` for the message), and dropped
  one unused `use std::iter` import along with similar small lint
  fixes across `request/`, `transaction/`, and `stats.rs`.
- `cargo fmt` re-ordered a couple of imports the previous commits got
  wrong (rustfmt's lexical-case order put `rand::Rng` before
  `rand::rng` again).

Local verification: `make check` succeeds end-to-end (`cargo check
--all --all-targets --features integration-tests` + `cargo fmt
--check` + `cargo clippy --all-targets --features integration-tests
-- -D clippy::all`).
`taiki-e/install-action@nextest` used a tag reference where the tag
name itself (`nextest`) was the install signal — the action interprets
the ref as a tool name. The previous commit pinned the action to a
commit SHA, which preserved the binary but lost the implicit "install
nextest" signal. Result: `cargo nextest` wasn't on `$PATH` and
`make unit-test` / `make integration-test-*` died with
`no such command: nextest`.

Pass `tool: nextest` as an explicit input on both install steps so the
action knows what to install when it's pinned by SHA.
The previous cleanup dropped `tests/integration_tests.rs`,
`tests/failpoint_tests.rs`, and `tests/common/` along with the dev-deps
they needed, because I conflated them with the example binaries.
That broke `make integration-test-*`: nextest filters for `txn_` /
`raw_` names found nothing in the lib's 38 unit tests, so the job
exited with code 4 ("no tests run").

What I removed didn't actually pull in unmaintained crates — the
ansi_term/atty/instant chain came from `clap 2` (CLI examples) and
`serial_test 0.5` (parking_lot 0.11), not from the integration tests
themselves. Restoring the test files with a narrowly-scoped
modernised dev-dep set keeps the CI verification useful without
re-introducing the RUSTSEC issues we just cleared.

Restored:
- `tests/integration_tests.rs`, `tests/failpoint_tests.rs`,
  `tests/common/{mod,ctl}.rs` from the upstream tree.
- The `[[test]]` table entries in `Cargo.toml`.
- Minimal dev-deps actually used by those files: `env_logger 0.11`
  (logger init), `reqwest 0.12` with `rustls-tls` (PD HTTP API in
  `ctl.rs`), `serde_json 1` (parsing the PD response), and
  `serial_test 3` (replaces 0.5; the `#[serial]` attribute is the
  same API but 3.x drops the `parking_lot 0.11` / `instant`
  dependency that triggered RUSTSEC-2024-0384).
- rand 0.8 -> 0.9 migration in the restored test code
  (`thread_rng()` -> `rng()`, `gen_range` -> `random_range`,
  `gen::<T>` -> `random::<T>`).

Verification: `make check` clean (cargo check + fmt + clippy). The
integration tests still require a live PD+TiKV cluster, which the
workflow spins up via `tiup playground` — that's the next signal.
@tobiemh
tobiemh changed the base branch from 0.3-surreal.x to main May 26, 2026 06:42
This is the first release from the SurrealDB-owned fork. We're no
longer tracking upstream `tikv/client-rust` versions — the crate is
published as `surrealdb-tikv-client`, which already disambiguates
from upstream's `tikv-client`, so the `-surreal.N` suffix is dropped
in favour of plain semver.

0.5.0 also sorts unambiguously above upstream's 0.4.0 publish from
2026-02-07 (which kept the same vulnerable dep stack as 0.3.0).
@tobiemh
tobiemh force-pushed the tobiemh/modernize-deps branch from 82f9e29 to 943db3b Compare May 26, 2026 06:45
@tobiemh
tobiemh marked this pull request as ready for review May 26, 2026 06:48
@tobiemh
tobiemh merged commit 025f13d into main May 26, 2026
7 checks passed
@tobiemh
tobiemh deleted the tobiemh/modernize-deps branch May 26, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant