Skip to content

network(ethp2p): enable with just ZEAM_ETHP2P=1 — auto-derive endpoints + per-node runtime TLS cert - #1049

Merged
ch4r10t33r merged 4 commits into
mainfrom
ethp2p-broadcast-adapter
Jul 18, 2026
Merged

network(ethp2p): enable with just ZEAM_ETHP2P=1 — auto-derive endpoints + per-node runtime TLS cert#1049
ch4r10t33r merged 4 commits into
mainfrom
ethp2p-broadcast-adapter

Conversation

@ch4r10t33r

@ch4r10t33r ch4r10t33r commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Continues #1045 (GitHub won't reopen a closed PR whose branch was force-pushed). Same branch, rebased onto current main, scoped to only the pieces not already merged via #1046/#1047 — now with the TLS cert generated at runtime (per the review feedback below).

The ethp2p adapter and env-config layer already landed on main; this adds what makes it turnkey.

What this adds (on top of main)

  • Auto-derived endpoints. buildEthp2pConfig derives the ethp2p listen address + static peers from the node's own libp2p QUIC addresses, shifted by ZEAM_ETHP2P_PORT_OFFSET (default +1, the ethlambda convention). Set ZEAM_ETHP2P=1 and it wires itself; explicit env still overrides.
  • Per-node TLS cert generated at runtimeEthLibp2p.generateAuxQuicCertPems() mints a fresh self-signed cert bound to the node's secp256k1 host identity via the same libp2p_tls_cert.generate facility the primary QUIC transport uses. The PEMs are written under the node's data dir (the adapter consumes cert/key by path — it has no in-memory PEM entry point) and overridable via ZEAM_ETHP2P_SERVER_CERT / _KEY.

Addressing the review

Invocation

docker build --build-arg ETHP2P=true -t zeam-ethp2p .
docker run -e ZEAM_ETHP2P=1 ...   # endpoints + per-node cert auto-generated

Doubly gated (-Dethp2p=true build AND ZEAM_ETHP2P); default build/test unaffected.

Validation

  • zig build -Dethp2p=true ✅ · default zig build ✅ · zig fmt --check
  • generateAuxQuicCertPems is a direct reuse of the proven startQuicTransport cert-gen path; memory ownership audited (listen/peers/cert/key paths freed after init; node_key retained by the engine, outlives the process).

…evnet cert

Revives #1045 on top of main (which already carries the ethp2p adapter via #1046
and the env-config layer via #1047). Adds the pieces that were only on the old
#1045 branch, so an operator can enable ethp2p with just `ZEAM_ETHP2P=1` — no
manual port/cert wiring:

- `buildEthp2pConfig` derives the ethp2p listen address and static peers from the
  node's OWN libp2p QUIC addresses, shifted by `ZEAM_ETHP2P_PORT_OFFSET`
  (default +1, the ethlambda "ethp2p = gossipsub port + 1" convention). Every
  field keeps explicit-env precedence (`ZEAM_ETHP2P_LISTEN`,
  `ZEAM_ETHP2P_STATIC_PEERS`, ...) over the derived default.
- Identity is the node's `node_key` (unique per node; retained by the RS engine
  and outlives the process — so the shared TLS cert below is NOT the peer id and
  cannot cause identity collisions).
- Bundled self-signed devnet TLS cert/key at `/app/resources/ethp2p/{cert,key}.pem`
  (shipped via the existing `COPY resources/`), overridable with
  `ZEAM_ETHP2P_SERVER_CERT` / `ZEAM_ETHP2P_SERVER_KEY`.
- Owned strings (listen_addr, static_peers) are freed via `freeEthp2pConfig`
  after `beam_node.init` — the adapter's `start` binds/dials synchronously and
  retains only `local_peer_id` (= long-lived `node_key`).

Deliberately does NOT include the old #1045 `pkgs/node/src/node.zig` hunk, which
predated and would revert the blocks_by_range sync-wedge fix (#1046).

Runtime activation unchanged and doubly gated: `-Dethp2p=true` build AND
`ZEAM_ETHP2P` truthy. Default build/test unaffected (adapter comptime-excluded);
`-Dethp2p=true` compiles.
…d keypair

Follow-up to review feedback: shipping a single self-signed cert/key for all
nodes is wrong — libp2p mints its QUIC TLS cert at runtime from the node's
identity, and ethp2p must do the same.

- Remove the committed resources/ethp2p/{cert,key}.pem.
- Add EthLibp2p.generateAuxQuicCertPems() — mints a fresh self-signed cert bound
  to the node's secp256k1 host identity via the SAME facility the primary libp2p
  QUIC transport uses (libp2p_tls_cert.generate + a fresh ephemeral cert key).
  Each call is unique per node and per process; nothing is shipped or shared.
- buildEthp2pConfig now, when it listens and no explicit cert env is set,
  generates the cert at startup and writes the PEMs under the data dir
  (ethp2p wants file paths — it has no in-memory PEM entry point). Explicit
  ZEAM_ETHP2P_SERVER_CERT / _KEY still override. Cert/key paths are heap-owned
  and freed by freeEthp2pConfig after beam_node.init.

The runtime cert can't be *literally* reused: the primary transport's cert is
generated in EthLibp2p.run(), which happens after beam_node.init sets up the
ethp2p listener — so at that point no cert exists yet. Generating a dedicated
per-node cert from the same host identity via the same facility gives the
identical security property (unique, runtime, never committed).

zig build -Dethp2p=true ✅ · default zig build ✅ · zig fmt --check ✅
@ch4r10t33r
ch4r10t33r force-pushed the ethp2p-broadcast-adapter branch from 8b7a904 to ee7da05 Compare July 17, 2026 20:32
@ch4r10t33r ch4r10t33r changed the title network(ethp2p): enable with just ZEAM_ETHP2P=1 — auto-derive endpoints from libp2p ports + bundled devnet cert network(ethp2p): enable with just ZEAM_ETHP2P=1 — auto-derive endpoints + per-node runtime TLS cert Jul 17, 2026
@ch4r10t33r

Copy link
Copy Markdown
Contributor Author

Updated per review: dropped the committed shared cert.pem/key.pem and now generate a per-node self-signed TLS cert at runtime from the node's host identity, using the same libp2p_tls_cert.generate facility the primary libp2p QUIC transport uses (EthLibp2p.generateAuxQuicCertPems). It can't literally reuse the libp2p cert because that one is minted in run(), after beam_node.init wires the ethp2p listener — so no cert exists yet at that point; a dedicated per-node cert from the same identity gives the same property. Overridable via ZEAM_ETHP2P_SERVER_CERT/_KEY.

The genesis peer list (nodes.yaml) always contains this node itself, so
the derived ethp2p static_peers included our own listen port. connect()
is synchronous and blocks start() in its handshake-poll loop; a self-dial
never completes because the server-side accept only runs later in tick().
That stalled the dial loop before it reached the real peers, so no ethp2p
peer ever connected. Skip any connect-peer whose libp2p QUIC port matches
our own.
@ch4r10t33r

Copy link
Copy Markdown
Contributor Author

Local 2-node ethp2p smoke — runtime cert validated

Ran a local 2-node smoke on lean-quickstart (branch ethp2p) with a locally-built zeam -Dethp2p=true, ZEAM_ETHP2P=1, genesis local-devnet/genesis.

Result: the runtime-generated per-node TLS cert completes a real, stable ethp2p QUIC handshake. Both nodes reached and held connected_peers=1 (verified via a temporary peer-count probe on BroadcastNode.peers, since removed): zeam_1 dialed zeam_0:9002, TLS negotiated against the freshly-minted generateAuxQuicCertPems cert (subject O=libp2p.io, ephemeral ECDSA-P256 key), the BCAST handshake completed, and each side registered the other as a peer. Sustained across 11 consecutive probes, both processes alive.

Fix included in this push (32dce7f)

Found and fixed a self-dial bug: the genesis peer list (nodes.yaml) always contains this node itself, so the derived static_peers included our own ethp2p listen port. connect() is synchronous and blocks start() in its handshake-poll loop; a self-dial never completes (the server-side accept only runs later in tick()), stalling the dial loop before it reached the real peers. Now skip any connect-peer whose libp2p QUIC port matches our own.

Known limitation (not blocking; follow-up)

Static-peer dials are synchronous and one-shot at start(), before either node enters its tick()/accept loop. connect() polls only dial_ep, not listen_ep, so a node blocked dialing cannot service inbound accepts. With simultaneous container starts, both nodes can block dialing each other before either accepts → no ethp2p peering, with no retry. The smoke worked by staggering (dialer launched only after the responder reached its accept loop) and pinning ZEAM_ETHP2P_STATIC_PEERS to just the peer. A robust deployment needs deferred/non-blocking dials + retry in zig_ethp2p (upstream follow-up).

@zclawz

zclawz commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

I reviewed #1049 adversarially. I am not approving yet; I found one blocking security issue.

Blocking finding:

  • pkgs/cli/src/node.zig:137 writes the generated ethp2p private-key PEM with createFile(io, path, .{ .truncate = true }), which uses Zig's default file permissions (0o666 before umask). That means the generated TLS private key can be world/group-readable under common umasks. Even though the cert key is runtime-generated and not the node host key, copying {database_path}/ethp2p/key.pem plus the signed cert is enough to impersonate that ethp2p QUIC listener for the cert lifetime. Since this PR's main security improvement is “no committed/shared keypair”, the runtime key needs owner-only permissions on creation, e.g. create/open with permissions = 0o600 (and ideally do not rely on umask).

Other checks looked fine:

  • Env-only enablement stays double-gated by -Dethp2p=true and ZEAM_ETHP2P.
  • Auto-derived listen/static-peer endpoints preserve explicit env overrides.
  • Self-entry skipping by own libp2p UDP port matches the genesis bootnode list shape.
  • generateAuxQuicCertPems reuses the libp2p TLS cert generation path and avoids committed key material.

Validation I ran:

  • zig fmt --check pkgs/cli/src/node.zig pkgs/network/src/ethlibp2p.zig
  • git diff --check origin/main...HEAD
  • Started /tmp/zig-0.16.0/zig build -Dethp2p=true --summary all with Rust on PATH; stopped it after the review blocker was confirmed, while it was still compiling Rust glue. No compile failure was observed before stopping.

Once the PEM private key is created with owner-only permissions, I expect this to be good to merge from my side.

@ch4r10t33r
ch4r10t33r merged commit 2d8ff56 into main Jul 18, 2026
14 checks passed
@ch4r10t33r
ch4r10t33r deleted the ethp2p-broadcast-adapter branch July 18, 2026 11:56
@ch4r10t33r

Copy link
Copy Markdown
Contributor Author

The blocking finding (world/group-readable ethp2p private key) is addressed in follow-up #1051: the key PEM is now created 0o600 (owner-only), verified on disk as -rw-------. Thanks @zclawz.

ch4r10t33r added a commit that referenced this pull request Jul 18, 2026
Review follow-up (zclawz on #1051): createFile's .permissions only applies
when the file is CREATED. A key.pem that an already-merged-#1049 node wrote
with the default 0o666 keeps that lax mode when this code rewrites it in
place (truncate=true), so the fix only covered fresh installs.

Explicitly setPermissions(0o600) on the key after open, while the file is
still empty (before the key bytes are written) so the secret is never
briefly present at a looser mode. The public cert passes null and keeps the
umask'd create default — it must not be force-set to 0o666 (world-writable).

Verified: a pre-existing key.pem at 0o666 is tightened to 0o600 in place
after the node runs (cert stays untouched).
ch4r10t33r added a commit that referenced this pull request Jul 18, 2026
* ethp2p: write the runtime TLS private key owner-only (0o600)

Per review (zclawz on #1049): the generated ethp2p private-key PEM was
written with default file permissions (0o666 before umask), leaving it
group/world-readable under common umasks. A copy of key.pem plus the
signed cert is enough to impersonate this node's ethp2p QUIC listener for
the cert lifetime, which defeats this PR's whole point (no shared/committed
keypair). Thread a permissions arg through ethp2pWritePem and create the
key with 0o600; the cert (public) keeps the default. 0o600 carries no
group/other bits so the file is owner-only regardless of umask.

Verified on disk: key.pem = -rw------- (600), cert.pem = -rw-r--r-- (644).

* ethp2p: enforce 0o600 on the key even when key.pem already exists

Review follow-up (zclawz on #1051): createFile's .permissions only applies
when the file is CREATED. A key.pem that an already-merged-#1049 node wrote
with the default 0o666 keeps that lax mode when this code rewrites it in
place (truncate=true), so the fix only covered fresh installs.

Explicitly setPermissions(0o600) on the key after open, while the file is
still empty (before the key bytes are written) so the secret is never
briefly present at a looser mode. The public cert passes null and keeps the
umask'd create default — it must not be force-set to 0o666 (world-writable).

Verified: a pre-existing key.pem at 0o666 is tightened to 0o600 in place
after the node runs (cert stays untouched).
@zclawz

zclawz commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Thanks for the update @ch4r10t33r! Good to see the key permissions hardened to 0o600 in #1051. 👍

@zclawz

zclawz commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Thanks for the heads up, @ch4r10t33r! Glad the key-permission fix landed in #10510o600 is the right posture for a private key. Nothing more needed on my end here. 🔒

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants