cabal
cabal copied to clipboard
haskell
curl error with `cabal upload -d` (regression in 3.14)
It looks like uploading documentation got broken:
$ cabal-3.12.1.0 upload -d /Users/abel/bin/src/test-framework/dist-newstyle/test-framework-0.8.2.2-docs.tar.gz
Uploading documentation
/Users/abel/bin/src/test-framework/dist-newstyle/test-framework-0.8.2.2-docs.tar.gz...
Documentation successfully uploaded for package candidate...
$ cabal-3.14.2.0 upload -d /Users/abel/bin/src/test-framework/dist-newstyle/test-framework-0.8.2.2-docs.tar.gz
Uploading documentation
/Users/abel/bin/src/test-framework/dist-newstyle/test-framework-0.8.2.2-docs.tar.gz...
Error: cabal-3.14.2.0: '/usr/bin/curl' exited with an error:
curl: (56) chunk hex-length char not a hex digit: 0xd
$ curl --version
curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.64.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe UnixSockets
(I also tried cabal-3.14.1.0/1, same problem.)
curl: (56) chunk hex-length char not a hex digit: 0xd
Does this look helpful?
~~https://github.com/haskell/cabal/issues/10920~~
Sorry, wrong link. Here you go:
- https://github.com/haskell/cabal/issues/10252
Is there a workaround for this issue?
Does cabal --http-transport=plain-http upload -d docs.tar.gz --publish work for you?
@Bodigrim : Thanks, this works indeed.
@ulysses4ever : Thanks for linking to #10252 . It shows the same symptoms, but it is reported against 3.12, whereas I only see the failure in 3.14.
I can confirm again that I am suffering from a regression in 3.14:
Uploading documentation
/Users/abel/bin/src/hackage-security/dist-newstyle/hackage-security-0.6.3.0-docs.tar.gz...
Running: /usr/bin/curl --config - 'https://hackage.haskell.org/package/hackage-security-0.6.3.0/candidate/docs' --request PUT --data-binary '@/Users/abel/bin/src/hackage-security/dist-newstyle/hackage-security-0.6.3.0-docs.tar.gz' --write-out '
%{http_code}' --user-agent 'cabal-install/3.14.2.0 (osx; aarch64)' --silent --show-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x-tar' --header 'Content-Encoding: gzip'
...
curl: (56) chunk hex-length char not a hex digit: 0xd
stdin input:
--anyauth
--user AndreasAbel:PASSWORD
CallStack (from HasCallStack):
withMetadata, called at src/Distribution/Simple/Utils.hs:447:12 in Cabal-3.14.2.0-inplace:Distribution.Simple.Utils
The -v3 for 3.12 is:
...
Running: /usr/bin/curl --config - 'https://hackage.haskell.org/package/hackage-security-0.6.3.0/candidate/docs' --request PUT --data-binary '@/Users/abel/bin/src/hackage-security/dist-newstyle/hackage-security-0.6.3.0-docs.tar.gz' --write-out '
%{http_code}' --user-agent 'cabal-install/3.12.1.0 (osx; aarch64)' --silent --show-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x-tar' --header 'Content-Encoding: gzip'
...
Documentation successfully uploaded for package candidate. You can now preview
the result at
'https://hackage.haskell.org/package/hackage-security-0.6.3.0/candidate'.
I now made the experiment of reverting #10089 and it indeed fixes the regression for me.
Running: /usr/bin/curl --config - 'https://hackage.haskell.org/package/hackage-security-0.6.3.0/candidate/docs' --request PUT --data-binary '@/Users/abel/bin/src/hackage-security/dist-newstyle/hackage-security-0.6.3.0-docs.tar.gz' --write-out '
%{http_code}' --user-agent 'cabal-install/3.15.0.0 (osx; aarch64)' --silent --show-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x-tar' --header 'Content-Encoding: gzip'
(However, in the light of #10252 there seems to be an underlying problem and likely #10089 just exposed it for me.)
Hackage server issue:
- https://github.com/haskell/hackage-server/issues/1389
Is there a workaround for this issue?
cabal --http-transport=wget upload works for me.
cabal upload is really in an absolutely disgraceful state.
hackage doc builders are broken and now I have to jump through hoops to be able to upload documentation for core libraries.
There's another workaround - the hackage web interface allows for directly uploading tarballs from your disk.
@hasufell Could you please refrain from using such hyperbolic language. All software has bugs, and this is one of many examples.
To summarise, --http-transport=plain-http can be passed to workaround the issue.
- It is reported that reverting #10089 works for some users but #10252 seems similar (and reproducible with earlier cabal versions)
- It seems that @andreasabel suggests that the server might be misconfigured in #10963
- @gbaz suggests that the problem may be in happstack (https://github.com/haskell/cabal/issues/10252#issuecomment-2787066289)
I am looking into how to fix the issue.
Could you please refrain from using such hyperbolic language.
The issues with cabal upload affect many users. They are very visible. They started surfacing almost a year ago and now also affect maintainers of core libraries negatively, where the absence of documentation is a major problem.
I think my language simply reflects the state of the user experience. This was not a statement related to volunteers and their efforts. We need to be able to express matters as they are and I will continue to call them out. I think we should all be able to differentiate between technology and people.
I am looking into how to fix the issue.
Terrific!
I have looked into this and it appears to be an issue with the nginx configuration. @bgamari is briefing the infrastructure team.
@hasufell Thanks for your thoughtful reply, I completely agree it's an important issue to fix, but that cause isn't helped with hyperbole and exaggeration. Your message shows me it is possible to express the same sentiment but in a more sensitive way.
I think there are two sides to the same coin. One side asks why is something so broken, the other side asks what can be done to make things better. Both people are motivated by the desire to improve a situation but the latter seems more positive to me.
What I have tried:
- Uploading documentation to
hackage.haskell.org, fails - Uploading documentation to
hackage-origin.haskell.orgfails - Uploading documentation to a local hackage instance (works)
You also need to use an upload which is quite "big", my documentation was 300kb, but it worked with --data 'test".
I was testing using the command:
curl --anyauth --user admin:admin 'http://127.0.0.1:8080/package/ghc-debug-client-0.7.0.0/docs' --request PUT --data-binary '@dist-docs/ghc-debug-client-0.7.0.0-docs.tar.gz' --write-out '\n%{http_code}' --user-agent 'cabal-install/3.14.2.0 (linux; x86_64)' --silent --show-error --location --header 'Accept: text/plain' --header 'Content-Type: application/x-tar' --header 'Content-Encoding: gzip' --trace-ascii response.trace
Sounds like nginx is configured for default max request size. When I was configuring the bridge there was a warning in the documentation to adjust it.
hackage has client_max_body_size 40M; -- i think the issue is that when the request is big enough, it turns on chunked encoding, and the issue is in how hackage handles the chunks...
I have done a fair bit of digging into this; indeed the problem is chunked encoding. Specifically, the problem appears to be an interaction between nginx's chunked-transfer implementation and that of hackage-server: The latter is able to handle directly-issued chunked requests without any trouble.
I have disabled chunking in our nginx configuration for the moment but this deserves more investigation.
Edsko reports that now the upload returns code 200 but documentation doesn't actually appear on the server. Can anyone reproduce that?
documentation doesn't actually appear on the server.
Does it appear when navigating to the actual url of a module's documentation for the new version? I no longer get the curl error but for at least months now I run into stale state all the time on the main package page (e.g. missing a new release which I can navigate to by url) and on a specific version's page (e.g. documentation appearing missing, but reachable by url).
Seems to fix itself after 5-15 minutes
@alexfmpe I think what you are describing is unrelated to the OP. It is an effect of agressive caching used on Hackage. The effect is, as you describe, that it takes long until you see the effect of changes. I also experience this. It is a bit annoying, esp. when (re-)uploading candidates and wanting to see the updated rendering (e.g. of the changelog...).
@andreasabel you're right that it's a different issue. I wonder, though, in the light of https://github.com/haskell/cabal/issues/11031, how much it's Hackage's responsibility vs. Cabal's.