Skip to content

🐛 Intermittent 524 on host with cloudflared tunnel #1691

Description

@MarcelloTheArcane

I'm getting intermittent but fairly frequent 524 timeouts when using a Cloudflare tunnel. It started about a month ago, and has been getting worse since.

It's particularly happening if I open a new tab to my site, or refresh an old tab. It's happening for all users, across multiple browsers (Chrome/Firefox).

Here's an example error log that I've captured while reproducing the issue:

$ cloudflared tail 1aacfce2-4fda-45b2-ad3d-4fc7a21f3a53
<...>
2026-07-07T10:58:57Z error http  {"connIndex":3,"error":"Incoming request ended abruptly: context canceled","ingressRule":0,"originService":"http://localhost:3000"}
2026-07-07T10:58:57Z error cloudflared Request failed {"connIndex":3,"dest":"<URL>","error":"Incoming request ended abruptly: context canceled","ip":"2606:4700:a8::9","type":"http"}
2026-07-07T10:58:57Z error cloudflared Failed to handle QUIC stream {"connIndex":3,"error":"stream 9261 canceled by remote with error code 0","ip":"2606:4700:a8::9"}
2026-07-07T10:58:57Z debug http upstream->downstream copy: read tcp [::1]:53656->[::1]:8080: use of closed network connection {"connIndex":3,"ingressRule":2,"originService":"http://localhost:8080"}
<...>

I've updated cloudflared to the latest version that I can:

$ cloudflared version
cloudflared version 2026.6.1 (built 2026-06-18-14:33 UTC)

It was installed from a package manager:

$ ls -la /usr/local/etc/cloudflared/
total 8
drwxr-xr-x 2 root root 4096 Jul  7 07:59 .
drwxr-xr-x 3 root root 4096 Jan  9  2025 ..
-rw-r--r-- 1 root root    0 Jul  7 07:59 .installedFromPackageManager

From the Troubleshoot page on QUIC timeouts, it doesn't seem like a firewall issue:

$ nc -uvz -w 3 2606:4700:a8::9 7844
Connection to 2606:4700:a8::9 7844 port [udp/*] succeeded!

Also, I am able to connect to the argotunnel.com URL:

$ dig SRV _v2-origintunneld._tcp.argotunnel.com

; <<>> DiG 9.18.39-0ubuntu0.24.04.5-Ubuntu <<>> SRV _v2-origintunneld._tcp.argotunnel.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49145
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;_v2-origintunneld._tcp.argotunnel.com. IN SRV

;; ANSWER SECTION:
_v2-origintunneld._tcp.argotunnel.com. 258 IN SRV 2 1 7844 region2.v2.argotunnel.com.
_v2-origintunneld._tcp.argotunnel.com. 258 IN SRV 1 1 7844 region1.v2.argotunnel.com.

;; Query time: 2 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Tue Jul 07 13:05:15 CEST 2026
;; MSG SIZE  rcvd: 156

Here's the diag output:

$ cloudflared tunnel diag 1aacfce2-4fda-45b2-ad3d-4fc7a21f3a53
2026-07-07T11:04:09Z INF If your instance is running in a Docker/Kubernetes environment you need to setup port forwarding for your application.
2026-07-07T11:04:09Z INF Selected server http://localhost:20241 starting diagnostic...
2026-07-07T11:04:09Z INF Collecting tunnel state...
2026-07-07T11:04:09Z INF Collected tunnel state.
2026-07-07T11:04:09Z INF Collecting system information...
2026-07-07T11:04:09Z INF Collected system information.
2026-07-07T11:04:09Z INF Collecting goroutine profile...
2026-07-07T11:04:09Z INF Collected goroutine profile.
2026-07-07T11:04:09Z INF Collecting heap profile...
2026-07-07T11:04:09Z INF Collected heap profile.
2026-07-07T11:04:09Z INF Collecting metrics...
2026-07-07T11:04:09Z INF Collected metrics.
2026-07-07T11:04:09Z INF Collecting log information...
2026-07-07T11:04:09Z INF Collected log information.
2026-07-07T11:04:09Z INF Collecting raw network information...
2026-07-07T11:04:09Z ERR Job: raw network information finished with error. error="error finishing traceroute: exit status 1"
2026-07-07T11:04:09Z INF Collecting network information...
2026-07-07T11:04:09Z ERR Job: network information finished with error. error="error finishing traceroute: exit status 1"
2026-07-07T11:04:09Z INF Collecting cli configuration...
2026-07-07T11:04:09Z INF Collected cli configuration.
2026-07-07T11:04:09Z INF Collecting configuration...
2026-07-07T11:04:09Z INF Collected configuration.
2026-07-07T11:04:09Z INF Collecting connectivity pre-checks...
2026-07-07T11:04:09Z INF Collected connectivity pre-checks.
2026-07-07T11:04:09Z INF Diagnostic file written: cloudflared-diag-2026-07-07T13-04-09+02-00.zip
2026-07-07T11:04:09Z WRN Diagnostic completed with one or more errors

Here's the output: cloudflared-diag-2026-07-07T13-04-09+02-00.zip

Originally, it reported that the UDP buffer size was too small, and directed me to quic-go's UDP buffer size wiki. I increased this to 7.5MB as per the wiki (since there was no indication from the diag run of an expected value), and subsequent runs of cloudflared tunnel diag don't show this warning.

Here are the buffer sizes:

$ sudo sysctl -a | grep net.core.wmem
net.core.wmem_default = 212992
net.core.wmem_max = 7500000
$ sudo sysctl -a | grep net.core.rmem
net.core.rmem_default = 212992
net.core.rmem_max = 7500000

Based on this StackOverflow question about seeing available UDP buffer space, I ran the following command which doesn't seem to show any drops:

$ sudo cat /proc/net/udp
   sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode ref pointer drops
 2319: 3600007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000   991        0 1655 2 ff437e5c88733600 0
 2319: 3500007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000   991        0 1653 2 ff437e5c88731680 0

I'm not sure what else I need to do to find the root cause here.

Expected behavior
I would like it to not time out.

Environment and versions

  • OS: Linux
  • Architecture: AMD64
  • Version: 6.8.0-134-generic

Metadata

Metadata

Assignees

No one assigned

    Labels

    Priority: NormalMinor issue impacting one or more usersType: BugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions