flathubbot is spamming the Buildserver with org.kde.kontact

[JakobDev]

I saw org.kde.kontact multiple times on the Buildserver ons serveral days. I wondred why and took a look at the PRs of org.kde.kontact. f-e-d-c currently creates a PR almost every time when it runs, which is hourly, what results in a lot of testbuilds. The Build fails, but the Buildserver did not recognize that, what causes that the Build is hanging. At the time of writing, there are currently 3 testbuilds of org.kde.kontact on the Buildserver, which may block other Apps. You should take a look at what causes this Spam.

[gasinvein]

In the log there are errors like this:

ERROR src.manifest: Failed to check archive akregator/akregator-22.04.2.tar.xz with AnityaChecker: Wrong content type 'text/plain' received from 'https://download.kde.org/stable/release-service/22.04.3/src/akregator-22.04.3.tar.xz'

It indicates a broken or outdated mirror. And updating this many sources at once makes hitting a broken mirror highly likely. This, in turn, causes the set of successfully updated sources to be randomized on each run, resulting in PR spam.

Honestly, I'm not sure what we can do about this. Hardcode some "primary" mirror instead of download.kde.org, if KDE provides one?

[tinywrkb]

Can we at least disable the flathubbot checks for it until the maintainer will update the app? I mean, it's still generating new PRs.

[gasinvein]

It can be disabled by setting "disable-external-data-checker": true in flathub.json. From the way you put it, I suppose the maintainers are AFK currently?

[JakobDev]

If f-e-d-c has a broken mirror in one run, but it worked and found a new version in a previous run, it can use the version from the previous run. That could stop such a Spam in the future.

[Tamaranch]

I have a duplicate PR and I guess it's the same problem: https://github.com/flathub/org.xfce.ristretto/pull/81 and https://github.com/flathub/org.xfce.ristretto/pull/82

ERROR src.manifest: Failed to check git libopenraw/libopenraw.git with AnityaChecker: Listing Git remote failed: Command '['git', 'ls-remote', '--exit-code', 'https://github.com/freedesktop/libopenraw.git']' returned non-zero exit status 128.

However in this case there is only one updated package and only one dependency whose url is broken, so I'm not sure if the above remark is still relevant:

It indicates a broken or outdated mirror. And updating this many sources at once makes hitting a broken mirror highly likely. This, in turn, causes the set of successfully updated sources to be randomized on each run, resulting in PR spam.

Note: I already changed the url on Anitya, so it should be fine soon: https://release-monitoring.org/project/7980/

[Tamaranch]

False alarm, in fact the first update was invalid.

[bcooksley]

I've been pointed to this issue in regards to the problem you are having with download.kde.org (Sysadmin here)

Please note that it is an instance of Mirrorbits (a mirror monitoring and redirection system), so as a first port of call f-e-d-c needs to correctly handle HTTP redirects. Based on the log printed above, my suspicion is that it does not - or at the very least poorly logs it's behaviour as we're not seeing which mirror you are talking to.

Note that as part of normal operations Mirrorbits monitors the mirrors via RSync and will only redirect you there if it has records of the file being present, with the correct timestamp (subject to drift tolerance) and size.

As steps forward to investigate this further can you please improve the logging in f-e-d-c to capture the actual redirected URL it is having issues with or at least make that diagnostic information available?

[wjt]

as a first port of call f-e-d-c needs to correctly handle HTTP redirects. Based on the log printed above, my suspicion is that it does not

It does follow redirects.

• or at the very least poorly logs it's behaviour as we're not seeing which mirror you are talking to.

The exception raised in this case indeed only includes the original URL, not the result of following any redirects.

As steps forward to investigate this further can you please improve the logging in f-e-d-c to capture the actual redirected URL it is having issues with or at least make that diagnostic information available?

The diagnostic information may be available in the debug output, but I opened #377 to include the real URL (as well as the original URL) in the message when this bad-content-type exception is raised.

[bcooksley]

Thanks, once that is known I can follow up with the mirror operator / investigate further. Given we have a *.tar.xz it is likely an older mirror that lacks the necessary mimetype information.

[wjt]

A change by @Alexander-Wilms just landed which causes the Content-Type to be ignored in favour of sniffing the content-type from the response. That should cause this problem to go away. It may be interesting to refresh #337 to warn when the sniffed content type does not match the stated content type, so that the mirror operator(s) in question can be identified.