Add http headers / TLS JA3 fingerptint to httpclient when fetching feedUrl

[josepowera]

trafficstars

Allow to add headers to outgoing request. For some sites we find, that they require specific headers, and then they work under curl. However in miniflux we can't add any headers, that would make httprequest more like comming from web browser.

This feed fails in miniflux: https://denar.mk/feed

however, when we run it in curl it goes thru.

curl 'https://denar.mk/feed' \
  -H 'authority: denar.mk' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'cache-control: max-age=0' \
  -H 'sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: document' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-site: none' \
  -H 'sec-fetch-user: ?1' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36' \
  --compressed

Our proposal is to allowed to add MAP of additinal key values parameters to headers (like accept-languages, ....) and in our example this also helps to go thru cloudflare protection. For simplicity this could also be written in JSON style field (it makes GUI much easier)

[josepowera]

Also allow to specify TLS JA3 fingerprint https://github.com/Danny-Dasilva/CycleTLS to allow to go thru cloudflare bot protections (https://developers.cloudflare.com/bots/concepts/ja3-fingerprint/) or nginx: https://github.com/fooinha/nginx-ssl-ja3 or haproxy: https://www.haproxy.org/download/2.5/doc/configuration.txt (X-SSL-JA3)

[qbunt]

also seeing this in latest miniflux for https://ilovetypography.com/blog/. I get the bot protection message, likely from cloudflare not seeing the required headers in the request.