Optimize HTTP/2 upstream backpressure - #74
Conversation
anyakushin
left a comment
There was a problem hiding this comment.
Hello, @xchacha20-poly1305, thanks for your effort and sorry for a long delay.
This PR has been reviewed and the team have some concerns about it:
- The session window is not respected by design: the TrustTunnel multiplexes unrelated TCP connections via different HTTP/2 streams, so these streams should be unrelated too. Respecting the session window would just result in re-coupling these connections and distort the backpressure. If there is one stream that advertises a big window and starves, it will fill the whole session window that will starve all other TCP connections. So this is an unacceptable change.
- The local data-source queue capacity limit is not required - the read from TUN is stopped if the remove window is 0 so this will be just an additional unnecessary protection that will never be triggered.
- The 4 MiB queue threshold change requires the evidences. I've done some bench tests and don't see any performance improvements even with relatively big BDP. Moreover, the 4MiB is big enough to exceed the network extension memory limit's for apple devices, so even with concrete evidences this can't be a hardcoded default for all platforms.
Thus, this PR can't be accepted in the current form, respecting session window is strongly unacceptable change by design and other changes require evidences.
Thanks again for your effort, any following work on this PR and finding the evidences would be very appreciated, i personally failed to find any myself.
P.s. husi is awesome
|
Thanks for the explanation. I agree with your concerns. I’m not deeply familiar with C++ and this part of TrustTunnel’s flow-control design, and my change is only based on observing a performance issue and trying to improve it. From your explanation, I understand why this approach is not acceptable for TrustTunnel’s design. I don’t plan to continue pushing this PR. I hope the TrustTunnel team can find a better optimization strategy that fits the codebase. |
Related Issue
Summary
Make HTTP/2 upstream back-pressure honor all three relevant limits — peer
session window, peer stream window, and local data-source queue capacity — so
callers stop over-committing bytes. Also split the TCP socket's single 128 KiB
constant into a 4 MiB queue threshold and a 256 KiB per-write cap to avoid
starving the upper layer on fast links.
Changes
core/src/http2_upstream.cpp:Http2Upstream::send()clampslengthtoavailable_to_send(id)and returns the actual bytes accepted instead ofunconditionally reporting
length.net/src/http2.cpp:http_session_available_to_write()returnsmin(session_window, stream_window, DATA_QUEUE_SIZE − pending)for non-zero stream IDs.data_source_add()rejects writes that would overflowDATA_QUEUE_SIZEwith
NGHTTP2_ERR_BUFFER_ERRORbefore touching the evbuffer.net/src/tcp_socket.cpp: ReplaceMAX_WRITE_BUFFER_LENwithMAX_WRITE_QUEUE_LEN(4 MiB, used bytcp_socket_available_to_write) andMAX_SINGLE_WRITE_LEN(256 KiB, passed tobufferevent_set_max_single_write).net/test/test_http2.cpp(+net/CMakeLists.txt): New unit test coveringboth stream-window and session-window back-pressure transitions.
common/src/event_loop.cpp,net/src/os_tunnel.cpp,tcpip/test/test_vpn_packet_pool.cpp: Replacestd::unique_ptr<T>{new T{}}with
std::make_unique<T>().Tests
net/test/test_http2.cppexercises stream-window exhaustion, queue-cap exhaustion, and recovery via
WINDOW_UPDATEon both the stream and the connection)make test)/benchon this PR to dispatch theendpoint-bench workflow (
.github/workflows/bench.yml), then paste theresults into the Benchmarks section below
Benchmarks
Need to run
/benchcommand. Skip for now.Checklist