getting 'HTTP Error 400: Bad Request' on submission

[tungol]

This is possibly a problem with musicbrainzngs. Possible cause is that my password uses a few unusual symbols - one or more of these may not be being escaped properly. It's not throwing an authentication error, however, which is what I'd expect looking at the code. On the other hand, I don't get an authentication error even when entering bogus credentials, that generates a 400: Bad Request as well. So I'm not sure what's going on there.

EDIT: The problem for this 400 error is a proxy. Another sources of 400 errors could be #67.

This is blocked on https://rt.cpan.org/Public/Bug/Display.html?id=84789 (upstream of MB server)

[JonnyJD]

Hm, non-ascii passwords, that can be a problem in my script, in musicbrainzngs, in the musicbrainz server (web service part) or even at another place (python?) I am working on a test script with some debug output to find out what actually happens.

What isrcsubmit version and/or package (isrcsubmit-*-mac.zip) are you using? What python version are you using? (python --version) Since you are on a Mac, this is probably some Python 2 version, but which? What version of python-musicbrainzngs are you using? The one shipped with isrcsubmit (see question above)?

I couldn't reproduce your "400: Bad Request", yet, but I get a HTTP Error 401: digest auth failed using python2 (2.7.3) and an uncaught UnicodeEncodeError: 'ascii' codec can't encode character\xe4in position 26: using python (3.3.1). (You should remove the actual characters/passwords when posting error message, mine is just a test password anyways)

[JonnyJD]

The test script is available at https://gist.github.com/JonnyJD/5429281. This just skips all the other steps and tries to submit isrcs (an empty dict actually), but adds a bit of debug output.

Please post the output of that script and python --version so I can see what exact problem you are having. My 401 problem might have a different cause (I just changed my password on test.musicbrainz.org).

[JonnyJD]

In case this isn't related (as in you also don't get the 400 error using my password_test.py) I need the output of isrcsubmit.py when this 400 error occurs. Inluding the disc ID and the ISRCs.

FYI: I am not sure the 401 I get with my testcript can be solved. In General HTTP (Digest) Auth (which is what we are using here) is not able to negotiate the encoding used for the username and password. So username and password are expected to be in ASCII. EDIT: See also http://stackoverflow.com/questions/702629/utf-8-characters-mangled-in-http-basic-auth-username for that (accepted answer) EDIT: I got it to work on Python 3 when patching pythons urllib.request with H = lambda x: hashlib.md5(x.encode("latin1")).hexdigest(). I don't think that is a solution that will be accepted by python. It doesn't work pre-encoding it in isrcsubmit.

[JonnyJD]

If possible, running isrcsubmit (current master) with --debug would help. I added some debug output for submit_isrcs.

[JonnyJD]

I opened a new ticket for the "401: Digest Auth failed" exception: #58 .

There is a ticket to expose the exact 400 error in alastair/python-musicbrainz-ngs#80. Without that it is a bit more guessing involved from my side.

[tungol]

The password is all ascii. I use 1password to randomly generate passwords, which includes ascii symbols, since it's a web service I imagined would need URL escaping - I could be wrong about that, I don't know how the MB API works. I'll run the tests you asked for now.

[tungol]

Python 2.7.2 isrcsubmit_version = "2.0.0-beta.1"

python packages installed with pip: musicbrainzngs==0.3 python-musicbrainz2==0.7.4 discid==0.4.0

libdiscid installed with brew: libdiscid: stable 0.4.1

discisrc from the isrcsubmit-1.0.1-mac package

it doesn't seem to matter what I put in usename and password:

./password_test.py ERROR: Entering submit_isrcs ERROR: Entering authenticate ERROR: Starting authentication

Please input your MusicBrainz username: [my username] Please input your MusicBrainz password:

ERROR: Leaving authenticate ERROR: Starting actual submission ERROR: Couldn't send ISRCs: caused by: HTTP Error 400: Bad Request

OR just entering user: test123 password: 123

/password_test.py ERROR: Entering submit_isrcs ERROR: Entering authenticate ERROR: Starting authentication

Please input your MusicBrainz username: test123 Please input your MusicBrainz password:

ERROR: Leaving authenticate ERROR: Starting actual submission ERROR: Couldn't send ISRCs: caused by: HTTP Error 400: Bad Request

[tungol]

and here's the most recent -dev version:

python isrcsubmit.py -d 1 --debug isrcsubmit 2.0.0-dev by JonnyJD for MusicBrainz

using discisrc CD drive #1 corresponds to 1 internally Warning: Unable to read the disc's media catalog number.

DiscID: kmtZWQ.s_5hxRGCGT4gxzK9MhfU- Tracks on disc: 19

Artist: Various Artists Release: The Silver Ticket: Terminal Sales Vol. VI

Warning: Unable to read the disc's media catalog number. MCN : Track 1 : USSUB1304801 Track 2 : USSUB1304802 Track 3 : USSUB1304803 Track 4 : USSUB1304804 Track 5 : USSUB1304805 Track 6 : USSUB1304806 Track 7 : USSUB1304807 Track 8 : USSUB1304808 Track 9 : USSUB1304809 Track 10 : USSUB1304810 Track 11 : USSUB1304811 Track 12 : USSUB1304812 Track 13 : USSUB1304813 Track 14 : USSUB1304814 Track 15 : USSUB1304815 Track 16 : USSUB1304816 Track 17 : USSUB1304817 Track 18 : USSUB1304818 Track 19 : USSUB1304819 found new ISRC for track 1: USSUB1304801 found new ISRC for track 2: USSUB1304802 found new ISRC for track 3: USSUB1304803 found new ISRC for track 4: USSUB1304804 found new ISRC for track 5: USSUB1304805 found new ISRC for track 6: USSUB1304806 found new ISRC for track 7: USSUB1304807 found new ISRC for track 8: USSUB1304808 found new ISRC for track 9: USSUB1304809 found new ISRC for track 10: USSUB1304810 found new ISRC for track 11: USSUB1304811 found new ISRC for track 12: USSUB1304812 found new ISRC for track 13: USSUB1304813 found new ISRC for track 14: USSUB1304814 found new ISRC for track 15: USSUB1304815 found new ISRC for track 16: USSUB1304816 found new ISRC for track 17: USSUB1304817 found new ISRC for track 18: USSUB1304818 found new ISRC for track 19: USSUB1304819

Do you want to submit? [y/N] y tracks2isrcs: {'14e521b8-08c9-4405-ad0a-495282b6cf76': u'USSUB1304803', 'd06b728b-1a7f-4369-bcde-30d189d07884': u'USSUB1304811', 'cba26480-b406-4235-9964-510dd3d3c24b': u'USSUB1304817', '9d70dc5d-640b-4d71-8e78-6441a4adedb7': u'USSUB1304808', 'af4b5fd0-50ff-443d-9e96-a6be19d2f6a6': u'USSUB1304813', '9685f2d2-c484-4a40-99c5-a373fa751f3e': u'USSUB1304809', '66e57e18-384a-4166-9eee-fc2c4479b53a': u'USSUB1304814', 'c8e7dcdd-50e4-457e-bba2-0c293aaa02b6': u'USSUB1304812', '69484878-94ca-4acd-a8d6-d52d535232a4': u'USSUB1304806', '78d439b3-add0-4e41-aa02-30ac38418fb6': u'USSUB1304801', '0388f725-8ba7-484a-a8a6-8c94810f3088': u'USSUB1304804', '78e6f29a-d1c6-4dd2-b073-016593c2faf2': u'USSUB1304810', '5eb809bd-94c0-4cc0-86e0-4526e2420a56': u'USSUB1304807', 'f878c8ac-3fa5-4ded-af02-f7646f89933f': u'USSUB1304818', 'b1d1004f-e0b9-4a58-914a-9f8391dea48b': u'USSUB1304802', '46ba86a8-e2a2-426a-a338-dca05ce08347': u'USSUB1304816', '6d30890d-0412-4211-a070-5311fd2f5d38': u'USSUB1304819', 'f9a8a9be-0780-42e7-8a01-31e8aa41647c': u'USSUB1304805', '8eafe934-a416-4286-b36e-d5618f7f6596': u'USSUB1304815'}

Please input your MusicBrainz username: [my username] Please input your MusicBrainz password:

ERROR: Couldn't send ISRCs: caused by: HTTP Error 400: Bad Request

[JonnyJD]

Thanks a lot for your tests. I made a couple semi-related changes to the code based on your test data.

However, I could reproduce your problem. The problem you seem to have is probably http://bugs.python.org/issue4140 You have a proxy in between musicbrainz.org and your client? I still have to figure some things out, but this looks like a musicbrainz server problem then.

When you can get around using a proxy things will work with current master (I added changes after your tests), but if you don't and things need to be fixed on MB server then this takes at least 2 weeks (due to the release cycle for MB server).

EDIT: This indeed doesn't seem to have anything to do with your unusual ascii characters, since the same error occurs using only alphanumeric characters.

[JonnyJD]

Reported the proxy problem to MusicBrainz-server with http://tickets.musicbrainz.org/browse/MBS-6185.

EDIT: FYI: If this needs fixes in a library MB-server uses, it can take quite some time until this is fixed.

[JonnyJD]

FYI: This has gone up all the way to catalyst (library/framework MusicBrainz uses): https://rt.cpan.org/Public/Bug/Display.html?id=84789

[tungol]

That did the trick - the proxy I was behind is just for ad-blocking purposes. I turned that off and it went through no problem. An obnoxious bug, but at least I managed to shake out a bunch of lower-hanging fruit as well.

[Freso]

Is this still an issue?