free-haaretz icon indicating copy to clipboard operation
free-haaretz copied to clipboard

Haaretz and TheMarker status code 403

Open maxim-sm opened this issue 4 years ago • 6 comments

Hi,

Thanks for the add-on! It seems that Haaretz (and TheMarker) are detecting the add-on and are throwing error 403 when trying to access any page from these domains. Same issue is occurring with Bypass Paywalls extension: https://github.com/iamadamdev/bypass-paywalls-chrome/issues/1004

Tested on Firefox: Desktop Version: 82 Mobile Version: 68.10 Add-On Version: 0.9.7

1

maxim-sm avatar Nov 02 '20 14:11 maxim-sm

An interesting finding from the issue linked in the OP. Apparently the issue is with the user-agent (Googlebot).

maxim-sm avatar Nov 02 '20 17:11 maxim-sm

@maxim-sm The problem is likely not with the user agent string, but rather with it (finally?) being validated, probably using reverse dns-lookup.

I've noticed that I only get an error when using the google-bot agent, whereas the Bing user agent (see https://github.com/almog/free-haaretz/commit/4ebbe85f3705cd196fdc12d88789ef0863b768fa) still bypasses the paywall. I also tried Duck Duck Go and Yandex but both aren't excluded from the paywall

Also, when I tested this locally with about:debugging (after failing to package and use the XPI), I saw that there are multiple warning regarding the manifest file, which may or may not be the reason why FF reports the XPI as corrupted.

almog avatar Nov 02 '20 17:11 almog

managed to create a working XPI (attached here as ZIP but it's same) by uploading it as an unlisted extension to addons.mozilla.org and downloading the signed version.

free_haaretz.zip

borges136 avatar Nov 03 '20 12:11 borges136

@borges136 thank you it works now :)

murad2019 avatar Nov 05 '20 00:11 murad2019

managed to create a working XPI (attached here as ZIP but it's same) by uploading it as an unlisted extension to addons.mozilla.org and downloading the signed version.

free_haaretz.zip

@borges136 Any way to download it as XPI on android? Firefox doesn't allow manual installation of XPIs from the device, only via links (at least on android).

yoavsapir avatar Nov 06 '20 12:11 yoavsapir

sorry @yoavsapir , i don't have an android device... try with this link maybe? https://srv-store5.gofile.io/download/pWlfyb/free_haaretz-0.9.7-fx.xpi

borges136 avatar Nov 06 '20 16:11 borges136