meeb
Failed to make HTTP request to https://bandcamp.com/: unknown status code response: 403
This may be something that's changed on my network, but at some point recently I started getting 403 errors when trying to sync. It'd been working great for quite some time. I've updated the cookie - my bandcamp account is all good.. I can download through the bandcamp website no problems.
So just wanted to check if it's just me seeing this?
2024-09-02 11:18:26,690 bandcamp [INFO] Located Bandcamp session in cookies: <redacted>...
2024-09-02 11:18:26,690 bandcamp [DEBUG] Making get request to https://bandcamp.com/
Traceback (most recent call last):
File "/Users/m/Projects/Tools/bandcampsync/env/bin/bandcampsync", line 55, in <module>
do_sync(cookies_path, cookies, dir_path, media_format, temp_dir, ign_patterns)
File "/Users/m/Projects/Tools/bandcampsync/env/lib/python3.11/site-packages/bandcampsync/__init__.py", line 19, in do_sync
bandcamp.verify_authentication()
File "/Users/m/Projects/Tools/bandcampsync/env/lib/python3.11/site-packages/bandcampsync/bandcamp.py", line 148, in verify_authentication
soup = self._request('get', url)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/m/Projects/Tools/bandcampsync/env/lib/python3.11/site-packages/bandcampsync/bandcamp.py", line 92, in _request
raise BandcampError(f'Failed to make HTTP request to {mask_sig(url)}: '
bandcampsync.bandcamp.BandcampError: Failed to make HTTP request to https://bandcamp.com/: unknown status code response: 403
I've not seen this myself so this might be tricky to debug. If you know a bit of Python if you could pop a print(response.text) just below this line: https://github.com/meeb/bandcampsync/blob/main/bandcampsync/bandcamp.py#L91 it would help. That should spam the HTML of the 403 page into the terminal so you'd at least be able to see what the error is, if they've put it in plain text in the error page that is.
I seem to be having the same problem - it started weeks ago but I'm just now looking into it. I added the print statement @meeb suggested plus printed the URL.
2024-09-02 07:28:38,832 media [INFO] Detected locally downloaded media: 1565971307 = /Users/navicore/Music/Bandcamp/media/Nendo Suzuki/[OTMN063] Nendo Suzukis Am
en Clayworks EP
2024-09-02 07:28:38,832 bandcamp [INFO] Located Bandcamp session in cookies: 1%09t%3A1725284518%0...
ejs start ---------
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>403 Forbidden</title>
</head>
<body>
<h1>Error 403 Forbidden</h1>
<p>Forbidden</p>
<h3>Error 54113</h3>
<p>Details: cache-bfi-krnt7300115-BFI 1725287319 3792609103</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
ejs url: https://bandcamp.com/
ejs -------- end
Traceback (most recent call last):
File "/Users/navicore/Music/Bandcamp/../../git/navicore/bandcampsync/bin/bandcampsync", line 55, in <module>
do_sync(cookies_path, cookies, dir_path, media_format, temp_dir, ign_patterns)
File "/Users/navicore/git/navicore/bandcampsync/venv/lib/python3.12/site-packages/bandcampsync/__init__.py", line 19, in do_sync
bandcamp.verify_authentication()
File "/Users/navicore/git/navicore/bandcampsync/venv/lib/python3.12/site-packages/bandcampsync/bandcamp.py", line 153, in verify_authentication
soup = self._request('get', url)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/navicore/git/navicore/bandcampsync/venv/lib/python3.12/site-packages/bandcampsync/bandcamp.py", line 97, in _request
raise BandcampError(f'Failed to make HTTP request to {mask_sig(url)}: '
bandcampsync.bandcamp.BandcampError: Failed to make HTTP request to https://bandcamp.com/: unknown status code response: 403
I can't replicate this but obviously something on Bandcamp's end is denying the requests for you. I've just pushed a v0.4.1 release which bumps the user agent used, give that a try and see if it helps. If it doesn't then I guess Bandcamp must be identifying your bandcampsync some other way.
Oh and thanks for the help @navicore - unfortunately that generic 403 from Varnish doesn't reveal any extra useful information.
@meeb understood - just posted the output to show what they show. frustrating!
v0.4.1 gives same 403
I'll look into it more when I can...
I'd try refreshing your cookies and trying it on a different IP if you have one available somewhere as a next step.
apologies for taking so long to get a chance to look into this further. i was grabbing the cookie from safari, so tried a fresh login on firefox but still get 403 (i'm on mac sequoia (release candidate))
i get the same error in varnish
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>403 Forbidden</title>
</head>
<body>
<h1>Error 403 Forbidden</h1>
<p>Forbidden</p>
<h3>Error 54113</h3>
<p>Details: cache-akl10333-AKL 1726268598 4101719767</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
i tried changing IP via a vpn and still get the same error (cache details are different, tho)
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>403 Forbidden</title>
</head>
<body>
<h1>Error 403 Forbidden</h1>
<p>Forbidden</p>
<h3>Error 54113</h3>
<p>Details: cache-chi-klot8100092-CHI 1726268811 1615303414</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
A quick search for error "Varnish" with "Error 54113" shows this might actually just be a CDN endpoint error to do with lack of cache? Do you have the ability to try a VPN (or alternative similar different geolocation) to test a different CDN endpoint? Basically try it from a different country and see if that works. bandcamp.com is fronted by Fastly so this might be something their end, either a misbehaving PoP or some sort of automated request / bot blocking feature.
Error 54113 is listed here:
https://www.fastly.com/documentation/guides/concepts/errors/
as "Error responses from our routing systems will either have an empty response body or will cite error 54113, a reference to Fastly's Autonomous System number.".
Edit: upon further reading this seems to possibly be caused by an unhandled error in VCL config uploaded to Fastly.
the fastly doc says "usually as a 503" error but we're both getting 403s (forbidden). but I agree, remembering they serve from a CDN might be a clue, thx.
also though, I tried through the tor network and got the same 403.
i've tried a variety of things, including using vpns to test different regions around the world and still get the same error..
but then during the weekend i tried the docker version and... it works fine! this is running on the same machine, using the same cookie file etc..
maybe it's a weird python dependency thing?
How weird. I run it in the container myself. I would agree the only logical assumption is it's something to do with your specific local install or Python library versions. What platform were you trying to run it on without the container?
I'm on macOS sequoia - tried on 2 different macs with the same result. i'd been on the sequoia betas leading up to its release and sadly wasn't able to check on sonoma :(
I'm experiencing it with sonoma 14.6.1 (23G93).
I've been meaning to compare the libs in the docker image to whatever is getting pulled in to my venv env. Will also try is on linux tomorrow...
Well, if any of you native macOS users want to debug and dump the headers being sent to find the exact differences in the request I'm happy to review it. Running natively doesn't use any sort of Apple privacy tunnel thing by default does it?
I'm seeing this on Linux - it looks like Bandcamp is intentionally blocking the requests package (or at least some non-browser clients - cURL still works). https://github.com/easlice/bandcamp-downloader/issues/35 is related.
I can reproduce like this:
$ python3 -c 'import requests; requests.get("https://bandcamp.com").raise_for_status()'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://bandcamp.com/
Interestingly, curl_cffi works fine:
$ python3 -c 'from curl_cffi import requests; print(requests.get("https://bandcamp.com").status_code)'
200
I do set the user agent to something not requests-y:
https://github.com/meeb/bandcampsync/blob/main/bandcampsync/config.py#L2
However I don't do much more than that. If Bandcamp (or more likely Fastly, their CDN) are starting to block some clients it's unfortunate but I'm likely not going engage in a perpetual arms race. The container currently works fine for me.
I would suspect this is some sort of machine learning based client detection based blocking which is why the reports on this are seemingly pretty random (something like https://docs.fastly.com/products/bot-management ).
I could add an option for external scraper services using API keys I suppose, but of course that adds complexity, requires external accounts to operate which would likely cost money as well as send your Bandcamp authentication credentials to a third party so that would be less than ideal.
I don't mind accepting some PRs if anyone wants test changes that work for them to make any tweaks to how I make basic HTTP requests:
https://github.com/meeb/bandcampsync/blob/main/bandcampsync/bandcamp.py#L75
But it's pretty much not going to be testable if it is an ML-based CDN client detection block. It's likely curl_cffi works fine for you at the moment because the combination of your IP and the request headers set hasn't reached some mystic threat score.
As there's no update to this and no way for me to debug the issue I'll close this shortly. If there's any updates to this please add them to the issue.
I'll close this for now. Please open a new issue if you continue to experience problems. Thanks!
I'm hitting this issue on Windows - I modified the script (with my very rudimentary Python skills) and get the same "Varnish cache server" error in the HTML.
Has there been any movement on this? Anyone else figure out a solution?
(As an aside, thank you @meeb for making this, even if Bandcamp are doing their best to make it not work!)
I'm not sure there's anything particularly do for this, it seems completely sporadic and unreproducable, affecting some people and not others. I would assume this issue would eventually resolve itself for you. You could also try regenerating your cookies to see if that helps.
Edit: and thanks for the kind words!
Yeah it's a shame (to go off topic briefly...) - Bandcamp seems to be actively trying to prevent any form of access other than via its own website/app, which are both very underpowered. They are walling themselves off, in an era when other platforms are integrating ever more widely. They're in a perfect position to be the music provider for self-hosted collections, but the amount of manual intervention necessary to maintain a collection at the moment makes it untenable.
In the meantime, if anyone else figures out a way around this (or even better, a way of making it more permanent and less likely to get blocked as well) I'm all ears!