haskell
`cabal upload` throwing curl errors
Describe the bug
+ cabal upload --publish --publish -d dist-docs.nENRAe/file-io-0.1.3-docs.tar.gz
hackage.haskell.org username: maerwald
hackage.haskell.org password:
Uploading documentation dist-docs.nENRAe/file-io-0.1.3-docs.tar.gz...
Error: cabal: '/usr/bin/curl' exited with an error:
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding
Happens reproducibly.
To Reproduce
cabal v2-haddock --haddock-for-hackage --enable-doc --haddock-options=--quickjumpcabal upload --publish -d dist-newstyle/file-io-0.1.3-docs.tar.gz
Expected behavior
cabal upload should work or have actual error handling.
System information
- Operating system: Fedora 39
cabal3.12.1.0,ghc9.8.2
Good. We should be catching more errors from auxiliary tools, for sure. But it's hard to do anytime meaningful if we don't understand the underlying issue.
Are you behind any kind of proxy? Cf. https://github.com/curl/curl/issues/6760
If we find what curl command cabal tried to execute, we could experiment further. Perhaps, running cabal with -v (-v3?) would reveal the exact command.
If we know the command, running the same command with --trace log.txt as suggested in the ticket above could bring an insight...
I just hit the same error while trying to upload documentation for a candidate release of happy-lib here.
~~FWIW, for me the error was that I hadn't yet uploaded the regular sdist of the candidate package.~~ This does not seem related; see below
Possibly relatedly, my Ubuntu system just received updates to both curl and wget. Since I don't do much uploading to Hackage (and in fact don't even have an account), I can't check to see if this introduced a new bug.
BTW, curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding is not an error we can do anything about (wtf does it even mean? That's a curl internal burp), so I don't see the issue: we can't do anything other than report it to the user and bail.
I think in my case it would have helped to say "Hint: This curl error can occur when uploading haddocks for a package before uploading the package."
TBH I think the technically correct thing here is to see if the server outputs error text before it runs its side of the transaction — but I'm not sure that's practical unless it is changed to always output text of some kind (which will of course break every older version of cabal-install). So this strikes me as a poorly-designed Hackage protocol which can't sensibly be dealt with except by the kind of post facto heuristics you suggested, which will always be dubious.
I triggered the error again this morning. See https://github.com/haskell/hackage-server/issues/1070. I think it would be great to sort this out; I don't think I made an error while typing in my password because it came from the clipboard (from my password manager), and that same password worked while uploading the code of the package candidate.
FWIW it just reproduced fully for me locally when trying to upload a candidate for one of my packages:
❯ cabal upload -v -d /home/artem/tmp/BNFC-meta/dist-newstyle/alex-meta-0.3.0.14-docs.
tar.gz
Warning: this is a debug build of cabal-install with assertions enabled.
hackage.haskell.org username: ArtemPelenitsyn
hackage.haskell.org password:
Uploading documentation
/home/artem/tmp/BNFC-meta/dist-newstyle/alex-meta-0.3.0.14-docs.tar.gz...
Running: /run/current-system/sw/bin/curl --config - 'https://hackage.haskell.org/package/alex-meta-0.3.0.14/candidate/docs' --request PUT --data-binary '@/home/artem/tmp/BNFC-meta/dist-newstyle/alex-meta-0.3.0.14-docs.tar.gz' --write-out '
%{http_code}' --user-agent 'cabal-install/3.15.0.0 (linux; x86_64)' --silent --show-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x-tar' --header 'Content-Encoding: gzip'
CallStack (from HasCallStack):
withMetadata, called at src/Distribution/Simple/Utils.hs:447:12 in Cabal-3.15.0.0-inplace:Distribution.Simple.Utils
Error: cabal: '/run/current-system/sw/bin/curl' exited with an error:
curl: (56) chunk hex-length char not a hex digit: 0xd
(the curl error is phrased differently but the error code is the same, so I assume it just depends on the curl version)
When I googled this error earlier, it often appeared in conjunction with newlines in some request header. 0xd (\r) seems to support that corellation
It may also depend on what's in any error message hackage-server might be throwing. It would be interesting to see the output of that curl command with --verbose.
@geekosaur I fail to run the copy-pasted curl command because of the --config - bit, which says that it will get configuration from stdin afaiu: it just hangs and waits for input.
If I remove the --config - bit, it fails:
❯ /run/current-system/sw/bin/curl 'https://hackage.haskell.org/package/alex-meta-0.3.
0.14/candidate/docs' --request PUT --data-binary '@/home/artem/tmp/BNFC-meta/dist-new
style/alex-meta-0.3.0.14-docs.tar.gz' --write-out '
%{http_code}' --user-agent 'cabal-install/3.15.0.0 (linux; x86_64)' --silent --show
-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x
-tar' --header 'Content-Encoding: gzip'
Error: No authorization provided
curl: (56) chunk hex-length char not a hex digit: 0xd
401⏎
When I run cabal with -v3 it gives away the stdin it provides curl with:
--anyauth
--user ArtemPelenitsyn:<my password>
Well, at least it seems like the server rejects the request as unauthorised, and curl reports multiple issues, one of which is the one that I got:
* Need to rewind upload for next request
* Ignoring the response-body
* chunk hex-length char not a hex digit: 0xd
* Illegal or missing hexadecimal sequence in chunked-encoding <------- This is the one I got
* Closing connection
* TLSv1.2 (OUT), TLS alert, close notify (256):
It seems a bit weird that there is a newline in
--write-out '
%{http_code}'
. can you try without? Edit: Oh, that may just be the width of your terminal emulator window...
@sgraf812
it seems like the server rejects the request as unauthorised
mm, which line says that?
weird that there is a newline
very true: my terminal is a little junky when it comes to breaking up lines so copy-pasting with it does not always work nicely.
I updated the gist after running without the spurious newline as you suggest.
@ulysses4ever, https://gist.github.com/ulysses4ever/602014e347df797f2f16a14edc0a9272#file-gistfile1-txt-L46. Note however that it's just doing an HTTP POST, so indeed it's not going to get that response until after the server receives it. The weird part there is that curl tries it without authentication first and then retries with auth (and there's where it tripped up).
@sgraf812 re: the newline in http code: it does come from how HttpTransport is implemented:
https://github.com/haskell/cabal/blob/5d5110a376f1c6b623d8c92ee9e9b48805458ea5/cabal-install/src/Distribution/Client/HttpUtils.hs#L553
of course, it's not clear to me why it's there. But at least it's not an artifact of copy-pasting...
Meanwhile this command can be used:
curl -X PUT \
--header "Authorization: X-ApiKey <API Token>" \
-H "Content-Type: application/x-tar" \
-H 'Content-Encoding: gzip' \
--data-binary "@dist-newstyle/docs/XXXX-0.1.0.0-docs.tar.gz" \
https://hackage.haskell.org/package/XXXX-0.1.0.0/docs
The workaround from @tusharad works also with --user Username:Password instead of using an API token.
And the following mutation of the workaround works for package candidates (sometimes?):
curl -X PUT \
--user MikolajKonarski:admin123 \
-H "Content-Type: application/x-tar" \
-H 'Content-Encoding: gzip' \
--data-binary "@dist-newstyle/Cabal-3.14.2.0-docs.tar.gz" \
https://hackage.haskell.org/package/Cabal-3.14.2.0/candidate/docs
You may need
curl -X PURGE https://hackage.haskell.org/package/Cabal-3.14.2.0/candidate
to see the docs without waiting forever.
We should really open a hackage-server issue. The one referenced above (https://github.com/haskell/hackage-server/issues/1070) has nothing to do with what we see here (that one is about .tar vs. .tgz mishmash, which also was fixed recently).
if its an issue with how the server parses newlines in headers, then likely the non-cabal fix will have to be not in hackage itself, but upstream in happstack.
This isn't going to be fixable on hackage-server -- its about curl args that don't affect the actual communication with the server.
looks like the newline dates to here: https://github.com/haskell/cabal/commit/9591d954be7cdc908caf4ea3face02a41d6337c9
The --write-out is to capture the resultant http code. Curl likely changed its behavior here. The issue the patch addressed was the http code didn't always have a newline. I suggest that we remove the newline before http_code (which new curl doesn't parse) and just fixup the processing in reading in the file with the resultant http status code.
@ulysses4ever wrote:
When I run cabal with -v3 it gives away the stdin it provides curl with:
--anyauth --user ArtemPelenitsyn:<my password>
The --anyauth points to #10920.
Looks like this won't be solved for 3.16, which is a pity. Hope for 3.18 then...