QRZ download runaway process

[stevendodd]

Just so you are aware I was doing a QRZ download today after the new update, the server went to 100% CPU and never recovered, there was no log file from today so no errors were reported. It was the second QRZ download that was manually triggered, relatively short time between them. I had to restart the VM, which resolved the issue for now.

On 7 May 2025, at 14:33, Peter Goodhall <notifications@github.com> wrote:

2.6.18

Repository: magicbug/Cloudlog · Tag: 2.6.18 · Commit: 54fae03 · Released by: magicbug

What's Changed

Rewritten HRD Logbook sync code it now does batch imports and has noticeable speed increases.[@magicbug]
Fixed bug that allowed white space in the callsign field when in contest mode [@magicbug]
Improved Polish Language support [from SP4BAO]
Full Changelog: 2.6.17...2.6.18

—
This release has 2 assets:

Source code (zip)
Source code (tar.gz)
Visit the release page to download them.

—
You are receiving this because you are watching this repository.
View it on GitHub or unsubscribe from all notifications for this repository.

[github-actions]

Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check your issue body.

Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.

Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.

[magicbug]

2GB isn't an awful lot depending on the size of a logbook, I'd check how you have MySQL or whatever db your using is setup and optimised, its quite processor-intensive for short bursts.

You'll really need to dig into your server configs to find out the overall cause of the spike but I no longer see such problems on my production / development machines

[stevendodd]

I don't have any problems generally, this is just a one off after the update yesterday. It's just an FYI.

[stevendodd]

It's just happened again tonight, it's not a spike. It is 100% CPU spin until I reboot the machine. It's not memory constrained at all and has not happened previously except for today.

[stevendodd]

It looks like I'm just not able to use QRZ download nothing from yesterday has been confirmed and every time I try to use it the CPU flies directly to 100% and stay there until I reboot the VM. You can see from the screenshot that it is the Apache process that is using all the CPU not mysql. I have an enabled verbose logging and there is nothing useful coming out of the logs

[github-actions]

Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check #3275 (comment).

Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.

Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.

[stevendodd]

I've just had a successful QRZ download and it downloaded everything, does it no longer work incrementally?

[stevendodd]

It downloaded everything but didn't update the matched QSO's

[stevendodd]

at least two of those matched on qrz.com

[github-actions]

Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check #3275 (comment).

Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.

Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.

[stevendodd]

The QRZ download ran again this morning, again it downloaded the whole set of QSOs from the beginning of my history, and again, none of them have been marked as matched in Cloudlog. I imagine something has gone wrong in the migration to the new way of downloading…

[github-actions]

Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check #3275 (comment).

Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.

Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.

[stevendodd]

Any idea how to diagnose this issue. The process sometimes completes but downloads the entire history and doesn't mark matched QSOs. More often than not the mysql process does some work and then the apache process continues at 100% cpu indefinitely until reboot. I gave the VM 4 CPUs and 4GB RAM and the same behaviour..

[stevendodd]

I've used the QRZ download page 3 times this morning since the latest update, each time the process completes and I get a list with a green or red confirmation on the right hand side and the CPU issue doesn't occur.

It downloads 970 QSOs which is an odd number as my QRZ logbook says: QSOs: 1399 Confirmed: 959
Once the process has completed, I would expect that the matched QSO's are marked as such in my logbook, but this is not okay and they remain unmatched

[stevendodd]

My analytics page in cloudlog has the following information

Year # of QSO's worked

1 2025 449
2 2024 745
3 2023 198

= 1392