integration/rpctest: improve rpctest harness by allocz · Pull Request #2571 · btcsuite/btcd

allocz · 2026-07-07T20:23:00Z

Change Description

The main motivation of this PR is to improve the process of writing integration
tests, currently there's no way to write complex tests in a simple way.

As discussed in #2560.

Changes

Debug event stream: connect to btcd and receive debug events.
Harness teardown: stops btcd returning the exit code, but allow stopping
without cleaning up node data, enabling the test of scenarios where btcd
exits with error or scenarios where the node should be restarted.
Harness start: allow the harness to be started again after teardown is
called.
Change Harness related procedure signatures: pass opts as a struct instead
of having to pass multiple zero values to get default initialization.

The debug event stream is replaced by a NOP implementation when the compile tag
debug is not defined, so that the compiler can optimize out all the debug event
broadcast calls. Also, the hidden flag --debugstream=host:port should be used
to start the event stream.

There's also bugfixes, the complete changelog is in the commit message.

Steps to Test

make unit
make unit-cover
make unit-race

Pull Request Checklist

Testing

Your PR passes all CI checks.
Tests covering the positive and negative (error paths) are included.
Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

The change is not insubstantial. Typo fixes are not accepted to fight bot spam.
The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
Commits follow the Ideal Git Commit Structure.
Any new logging statements use an appropriate subsystem and logging level.

📝 Please see our Contribution Guidelines for further guidance.

allocz · 2026-07-12T00:02:04Z

For now I think that we achieved what was proposed. Ready for review again. Thanks to @TechLateef for providing several fixes and improvements.

TechLateef

nit: Since this PR touches a number of different areas, would it make sense to split it into a few logically grouped commits? I think it would make the review a bit easier to follow and align with the project's recommendation of keeping changes in small, reviewable commits.

allocz · 2026-08-03T18:34:15Z

nit: Since this PR touches a number of different areas, would it make sense to split it into a few logically grouped commits? I think it would make the review a bit easier to follow and align with the project's recommendation of keeping changes in small, reviewable commits.

Yes, it makes sense.

The usage of option structs allows cleaner calls when the default behavior is needed and also allows future extensions without breaking the API.

allocz · 2026-08-11T17:57:19Z

@TechLateef, I've improved the commit history, as requested.

rpctest.New was removed and rpctest.New2 renamed to rpctest.New, fixed broken tests due to the API change.

With the signal option, we make it possible to skip sending the shutdown signal to the node, allowing the test of cases where we expect the node shutting down by itself.

Node restart support was implemented by creating the node instance on harness SetUp instead of New, this way we can also run the same node several times with a different set of arguments, allowing more test cases to happen. An option `NoRPCClientAndWallet` was added to simplify test cases where for some reason the RPC Client won't connect to the instance. Options `SkipCleanup` and `NoSignal` added to tear down procedure, enabling the node to be restarted while the state is kept and also allowing the test of cases where the node should shutdown by itself instead of being shutdown upon receiving a signal. Tests covering node restart and exit error code implemented and added to the harness test table.

There are cases where the RPC will work, but the wallet would not be able to finish sync up to best height, or we don't have reasons to wait for the sync to finish. Setting `NoWalletWait` to true allow us to skip waiting the wallet and speed up tests, or even avoid blocking forever.

Previously, each time harness setup was executed, a new goroutine was launched to handle the wallet updates, but the channel keeping the goroutine alive was never closed, causing goroutine leakage. The memwallet implementation was changed and now the memwallet goroutine is stopped when harness teardown is called. A test was added to assert that there's no goroutine leaks between harness setup and teardown.

The debug stream allows the register of debug events which are broadcasted to debug clients. The `debug` tag switches between the real and the nop implementation, so that there's no cost of sending debug events in production.

Now, when btcd is compiled with the `debug` tag and the flag --debugstream=<host:port> is passed to btcd, the debug stream server will be started, allowing btcd to broadcast debug events. When the harness option `DebugHandler` callback is set, the harness will connect with the debug stream, and the debug events can be handled by the callback.

The change allow us to see the name and status of the tests running, while also allowing to filter the tests by regexp instead of having to comment when we want to test a single test case in isolation. TestHarness previouly contained lines not wrapped to 80 columns.

Since the blocks are generated only when both CreateTestChain and NumMatureOutputs are set, there's no obvious reason to keep both. The code was simplified by ignoring the CreateTestChain option.

The previous commit made CreateTestChain option useless, so this commit cleans up the codebase by effectively removing the option and all the references to it.

allocz · 2026-08-19T14:00:09Z

I think that the current PR is doing too many things, causing difficulties to review the changes. So I'll close this PR and open smaller ones.

allocz force-pushed the improve_rpctest_harness branch from 14d655f to e5dfa5d Compare July 8, 2026 11:17

allocz mentioned this pull request Jul 8, 2026

[FEATURE] Implement stopatheight #2534

Open

7 tasks

TechLateef reviewed Jul 10, 2026

View reviewed changes

Comment thread integration/rpctest/rpc_harness.go Outdated

TechLateef reviewed Jul 10, 2026

View reviewed changes

Comment thread integration/rpctest/node.go

TechLateef reviewed Jul 10, 2026

View reviewed changes

Comment thread debugstream/client.go Outdated

TechLateef reviewed Jul 10, 2026

View reviewed changes

Comment thread debugstream/client.go Outdated

allocz force-pushed the improve_rpctest_harness branch from 74ffcc5 to 466eeb7 Compare July 10, 2026 23:37

allocz mentioned this pull request Jul 10, 2026

btcd: fix nonzero exit code on sigint when indexing cfindex #2557

Open

7 tasks

TechLateef reviewed Jul 11, 2026

View reviewed changes

Comment thread debugstream/common.go Outdated

allocz force-pushed the improve_rpctest_harness branch 2 times, most recently from e05176c to f4b587b Compare July 11, 2026 23:39

allocz force-pushed the improve_rpctest_harness branch 4 times, most recently from fb75252 to e7fe6fd Compare July 12, 2026 03:06

TechLateef suggested changes Aug 1, 2026

View reviewed changes

rpctest: use option structs to create, setup and teardown harness

044ae3f

The usage of option structs allows cleaner calls when the default behavior is needed and also allows future extensions without breaking the API.

allocz force-pushed the improve_rpctest_harness branch from e7fe6fd to de122dc Compare August 11, 2026 17:53

allocz force-pushed the improve_rpctest_harness branch from 4814d98 to 0694cb1 Compare August 11, 2026 20:00

allocz added 8 commits August 11, 2026 20:14

integration,rpctest: replace rpctest.New by rpctest.New2

f6dcea1

rpctest.New was removed and rpctest.New2 renamed to rpctest.New, fixed broken tests due to the API change.

rpctest,integration: replace harness SetUp by SetUp2

72f5420

rpctest,integration: replace harness teardown by teardown2

3f4f1b7

rpctest: do not ignore command error on node stop

f54a84c

rpctest: add signal option to node shutdown

ef0e482

With the signal option, we make it possible to skip sending the shutdown signal to the node, allowing the test of cases where we expect the node shutting down by itself.

rpctest: do not skip cleanup when node stops with error

26d999d

allocz added 6 commits August 11, 2026 20:14

debugstream: implemented server and client

0edb830

The debug stream allows the register of debug events which are broadcasted to debug clients. The `debug` tag switches between the real and the nop implementation, so that there's no cost of sending debug events in production.

rpctest: ignore redundant create test chain harness option

c8a31d2

Since the blocks are generated only when both CreateTestChain and NumMatureOutputs are set, there's no obvious reason to keep both. The code was simplified by ignoring the CreateTestChain option.

integration,rpctest: remove unused create test chain option

56762b2

The previous commit made CreateTestChain option useless, so this commit cleans up the codebase by effectively removing the option and all the references to it.

allocz force-pushed the improve_rpctest_harness branch from 0694cb1 to 56762b2 Compare August 11, 2026 20:15

allocz closed this Aug 19, 2026

allocz mentioned this pull request Aug 19, 2026

Improve RPC Harness API #2594

Open

7 tasks

Navigation Menu

Uh oh!

integration/rpctest: improve rpctest harness - #2571

integration/rpctest: improve rpctest harness#2571
allocz wants to merge 15 commits into
btcsuite:masterfrom
allocz:improve_rpctest_harness

allocz commented Jul 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

allocz commented Jul 12, 2026 •

edited

Loading

Uh oh!

TechLateef left a comment

Uh oh!

allocz commented Aug 3, 2026

Uh oh!

allocz commented Aug 11, 2026

Uh oh!

allocz commented Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

allocz commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Description

Steps to Test

Pull Request Checklist

Testing

Code Style and Documentation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

allocz commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TechLateef left a comment

Choose a reason for hiding this comment

Uh oh!

allocz commented Aug 3, 2026

Uh oh!

allocz commented Aug 11, 2026

Uh oh!

allocz commented Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

allocz commented Jul 7, 2026 •

edited

Loading

allocz commented Jul 12, 2026 •

edited

Loading