magicbug
QRZ download runaway process
Just so you are aware I was doing a QRZ download today after the new update, the server went to 100% CPU and never recovered, there was no log file from today so no errors were reported. It was the second QRZ download that was manually triggered, relatively short time between them. I had to restart the VM, which resolved the issue for now.
On 7 May 2025, at 14:33, Peter Goodhall <[email protected]> wrote:
Repository: magicbug/Cloudlog · Tag: 2.6.18 · Commit: 54fae03 · Released by: magicbug
What's Changed
Rewritten HRD Logbook sync code it now does batch imports and has noticeable speed increases.[@magicbug] Fixed bug that allowed white space in the callsign field when in contest mode [@magicbug] Improved Polish Language support [from SP4BAO] Full Changelog: https://github.com/magicbug/Cloudlog/compare/2.6.17...2.6.18
— This release has 2 assets:
Source code (zip) Source code (tar.gz) Visit the release page to download them.
— You are receiving this because you are watching this repository. View it on GitHub or unsubscribe from all notifications for this repository.
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check your issue body.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
2GB isn't an awful lot depending on the size of a logbook, I'd check how you have MySQL or whatever db your using is setup and optimised, its quite processor-intensive for short bursts.
You'll really need to dig into your server configs to find out the overall cause of the spike but I no longer see such problems on my production / development machines
I don't have any problems generally, this is just a one off after the update yesterday. It's just an FYI.
It's just happened again tonight, it's not a spike. It is 100% CPU spin until I reboot the machine. It's not memory constrained at all and has not happened previously except for today.
It looks like I'm just not able to use QRZ download nothing from yesterday has been confirmed and every time I try to use it the CPU flies directly to 100% and stay there until I reboot the VM. You can see from the screenshot that it is the Apache process that is using all the CPU not mysql. I have an enabled verbose logging and there is nothing useful coming out of the logs
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check https://github.com/magicbug/Cloudlog/issues/3275#issuecomment-2867293311.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
I've just had a successful QRZ download and it downloaded everything, does it no longer work incrementally?
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check https://github.com/magicbug/Cloudlog/issues/3275#issuecomment-2870027948.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
The QRZ download ran again this morning, again it downloaded the whole set of QSOs from the beginning of my history, and again, none of them have been marked as matched in Cloudlog. I imagine something has gone wrong in the migration to the new way of downloading…
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check https://github.com/magicbug/Cloudlog/issues/3275#issuecomment-2871358665.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
Any idea how to diagnose this issue. The process sometimes completes but downloads the entire history and doesn't mark matched QSOs. More often than not the mysql process does some work and then the apache process continues at 100% cpu indefinitely until reboot. I gave the VM 4 CPUs and 4GB RAM and the same behaviour..
I've used the QRZ download page 3 times this morning since the latest update, each time the process completes and I get a list with a green or red confirmation on the right hand side and the CPU issue doesn't occur.
It downloads 970 QSOs which is an odd number as my QRZ logbook says: QSOs: 1399 Confirmed: 959 Once the process has completed, I would expect that the matched QSO's are marked as such in my logbook, but this is not okay and they remain unmatched
My analytics page in cloudlog has the following information
Year # of QSO's worked
1 2025 449 2 2024 745 3 2023 198
= 1392
Just an update, I've done a little bit further analysis and when the CPU issue occurs the QRZ upload file is incomplete and remains in the upload directory. When the process does complete the upload file is unlinked but the QSO are still not matched.
By the way, congratulations on releasing Zenith it looks interesting. I might give it a go with the QTrigdoppler python app and my 9700!
My previous observation about the CPU issue occurring when the QRZ upload file is incomplete was not correct, it happens even when I get a completed response from QRZ with over 1500 QSL in it.
I completely reinstalled the OS, lamp stack and cloudlog as per the instructions here https://github.com/magicbug/Cloudlog/wiki/Installation this afternoon, imported my logbook and encountered the same problem.
I've run the virtual machine on a couple of different hardware options, including a N100 CPU so it's relatively new hardware; memory has not a problem it doesn't even get close to running out.
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check https://github.com/magicbug/Cloudlog/issues/3275#issuecomment-3092542949.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
Uh oh! @stevendodd, the image you shared is missing helpful alt text. Check https://github.com/magicbug/Cloudlog/issues/3275#issuecomment-3092542949.
Alt text is an invisible description that helps screen readers describe images to blind or low-vision users. If you are using markdown to display images, add your alt text inside the brackets of the markdown image.
Learn more about alt text at Basic writing and formatting syntax: images on GitHub Docs.
Please see https://github.com/magicbug/Cloudlog/pull/3304 this is basically best case now