wallabag
Support metarefresh to work with Anubis
Anubis is a tool to protect a website against aggressive (AI) crawlers. It works by providing a JS challenge that can be easily solved by a browser but not a bot doing massive requests. Sadly, it also broke friendly bots like Wallabag.
The good news is that the latest release of Anubis added no-js challenges using metarefresh.
You receive an header metarefresh with a number of seconds to wait. After X seconds, you can fetch the article.
Is your feature request related to a problem? Please describe. Can not fetch content of website protected by Anubis (e.g. https://wiki.freecad.org)
Describe the solution you'd like
Check for metarefresh presence in headers and wait for this number of seconds before retrying.
Describe alternatives you've considered Using extensions like Wallabager is an alternative (but not possible in all contexts).
Additional context
