Unable to fetch 2 independent feeds since update to v2.1.0
Hello there,
first of all, thanks for developing Miniflux, and especially for adding in the accessibility improvements in v2.1.0. Its much appreciated!
I just noticed however that two feeds which worked before upgrading to 2.1.0 are now showing a "Access forbidden" error when trying to fetch via Miniflux. I might be wrong, but the only two changelog entries that might be related to this seem to be the "Change default Accept header when fetching feeds" and the http/2 change. Is there anything I can do about that?
The two feeds are the one from https://www.anime2you.de/ and the rss feed from https://www.volksverpetzer.de/
Thanks alot in advance!
I've tried the Volksverpetzer blog and it looks like they are blocking based on the User Agent?
$ curl -I https://www.volksverpetzer.de/feed/
HTTP/2 403
$ curl -I -H "User-Agent: I am not cURL LOL" https://www.volksverpetzer.de/feed/
HTTP/2 200
but that's strange because it shouldn't have worked also before right?
I know that anime2you didn't work before 2.0.50 (I think, somewhere around that version), but definitely worked with 2.0.51 again, and it stopped working with 2.1.0 out of nowhere. I wasn't subscribed to Volksverpetzer for so long, but it did work flawlessly with 2.0.51 and maybe even before. If the User Agent however didn't change with the most recent version bump I don't know what might be causing this behaviour.
About anime2you.de I see that requests are blocked by Cloudflare. This request returns some Cloudflare garbage:
$ curl -H "User-Agent: hahaha" https://www.anime2you.de/feed/
HTTP/2 520
server: cloudflare
The same request with a valid Firefox user Agent is successful:
curl 'https://www.anime2you.de/feed/' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0'
Seems that in your case the ability to control the User-Agent header could help.
I just tried to edit my Miniflux subscription of anime2you and added this as user agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0
I however still get the forbidden error in Miniflux.
I'm also seeing HTTP 403 responses for all subreddit feeds since the update, e.g. for https://old.reddit.com/r/programming.rss
I might be wrong, but the only two changelog entries that might be related to this seem to be the "Change default
Acceptheader when fetching feeds" and the http/2 change. Is there anything I can do about that? The two feeds are the one from https://www.anime2you.de/ and the rss feed from https://www.volksverpetzer.de/
It's coming from the ForceAttemptHTTP2=true added in this PR https://github.com/miniflux/v2/pull/2257 which was supposed to fix #2255. Removing the option works for these 2 feeds. It might be worth to make it configurable.
I had one feed with the issue and it was driving me nuts. Tried replacing Agent string without luck but finally searched GH issues and found this. I'm glad it's already fixed. Finger crossed for the next release :-)