v2 icon indicating copy to clipboard operation
v2 copied to clipboard

Unable to fetch 2 independent feeds since update to v2.1.0

Open Timtam opened this issue 2 years ago • 6 comments

Hello there,

first of all, thanks for developing Miniflux, and especially for adding in the accessibility improvements in v2.1.0. Its much appreciated!

I just noticed however that two feeds which worked before upgrading to 2.1.0 are now showing a "Access forbidden" error when trying to fetch via Miniflux. I might be wrong, but the only two changelog entries that might be related to this seem to be the "Change default Accept header when fetching feeds" and the http/2 change. Is there anything I can do about that? The two feeds are the one from https://www.anime2you.de/ and the rss feed from https://www.volksverpetzer.de/

Thanks alot in advance!

Timtam avatar Feb 19 '24 15:02 Timtam

I've tried the Volksverpetzer blog and it looks like they are blocking based on the User Agent?

$ curl -I https://www.volksverpetzer.de/feed/
HTTP/2 403

$ curl -I -H "User-Agent: I am not cURL LOL" https://www.volksverpetzer.de/feed/
HTTP/2 200

but that's strange because it shouldn't have worked also before right?

jman-schief avatar Feb 19 '24 20:02 jman-schief

I know that anime2you didn't work before 2.0.50 (I think, somewhere around that version), but definitely worked with 2.0.51 again, and it stopped working with 2.1.0 out of nowhere. I wasn't subscribed to Volksverpetzer for so long, but it did work flawlessly with 2.0.51 and maybe even before. If the User Agent however didn't change with the most recent version bump I don't know what might be causing this behaviour.

Timtam avatar Feb 19 '24 20:02 Timtam

About anime2you.de I see that requests are blocked by Cloudflare. This request returns some Cloudflare garbage:

$ curl -H "User-Agent: hahaha" https://www.anime2you.de/feed/
HTTP/2 520
server: cloudflare

The same request with a valid Firefox user Agent is successful:

curl 'https://www.anime2you.de/feed/' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0'

Seems that in your case the ability to control the User-Agent header could help.

jman-schief avatar Feb 19 '24 21:02 jman-schief

I just tried to edit my Miniflux subscription of anime2you and added this as user agent:

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0

I however still get the forbidden error in Miniflux.

Timtam avatar Feb 19 '24 21:02 Timtam

I'm also seeing HTTP 403 responses for all subreddit feeds since the update, e.g. for https://old.reddit.com/r/programming.rss

moppman avatar Feb 19 '24 21:02 moppman

I might be wrong, but the only two changelog entries that might be related to this seem to be the "Change default Accept header when fetching feeds" and the http/2 change. Is there anything I can do about that? The two feeds are the one from https://www.anime2you.de/ and the rss feed from https://www.volksverpetzer.de/

It's coming from the ForceAttemptHTTP2=true added in this PR https://github.com/miniflux/v2/pull/2257 which was supposed to fix #2255. Removing the option works for these 2 feeds. It might be worth to make it configurable.

fguillot avatar Feb 22 '24 05:02 fguillot

I had one feed with the issue and it was driving me nuts. Tried replacing Agent string without luck but finally searched GH issues and found this. I'm glad it's already fixed. Finger crossed for the next release :-)

woj-tek avatar Mar 03 '24 20:03 woj-tek