hackage-server icon indicating copy to clipboard operation
hackage-server copied to clipboard

"Unexpected content-type" when uploading gzipped docs

Open dpwiz opened this issue 3 years ago • 18 comments

When running cabal haddock --haddock-for-hackage it results in a .tar.gz file being created.

But trying to upload it (chrome, firefox) I'm getting the error:

Unexpected content-type For this resource the content-type was expected to be application/x-tar, rather than application/gzip

And have to undo the gzip wrapping the cabal did for me before uploading it again.

This is annoying and wasteful.

dpwiz avatar May 10 '22 21:05 dpwiz

@dpwiz : Thanks for the bug report. Please provide full details that make the bug easily reproducible. (Think of a script that can be run to trigger the bug, without making permanent modifications to Hackage.)

andreasabel avatar May 11 '22 05:05 andreasabel

cabal haddock --haddock-for-hackage hello

HACKAGE_API_KEY=...............................
PACKAGE_VER=hello-0.1.0.0
curl \
  --header "Authorization: X-ApiKey $HACKAGE_API_KEY" \
  -F_method=PUT \
  -F_transform=file2raw \
  -F_file=@dist-newstyle/$PACKAGE_VER-docs.tar.gz \
  https://hackage.haskell.org/package/$PACKAGE_VER/docs

On a related note, I'd prefer having cabal haddock --upload or something like that rather than doing all of this manually.

dpwiz avatar May 11 '22 12:05 dpwiz

There is 'cabal upload -d' for that purpose. Does this work for you?

andreasabel avatar May 11 '22 14:05 andreasabel

Not quite....

Uploading documentation
dist-newstyle/package-name-0.1.0.0-docs.tar.gz...
cabal: '/usr/bin/curl' exited with an error:
Warning: -:2: warning: '--user' uses unquoted whitespace in the line that may
Warning: cause side-effects!
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding

dpwiz avatar May 11 '22 14:05 dpwiz

This looks like a bug in cabal. If possible, can you please search for this or similar in the cabal issues? I had a problem with curl and cabal once. Might also be OS specific.

andreasabel avatar May 11 '22 14:05 andreasabel

Meanwhile, I'd be glad for Hackage to allow me upload gzipped tarballs from the browser (:

dpwiz avatar May 11 '22 15:05 dpwiz

On macOS, I am getting e.g.

$ cabal upload -d /Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz -v
Uploading documentation
/Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz...
Running: /usr/local/opt/curl/bin/curl --config 
- 'https://hackage.haskell.org/package/HsYAML-aeson-0.2.0.1/candidate/docs' 
--request PUT 
--data-binary '@/Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz' 
--write-out '%{http_code}' 
--user-agent 'cabal-install/3.7 (osx; x86_64)' 
--silent --show-error --location 
--header 'Accept: text/plain' 
--header 'Content-Type: application/x-tar' 
--header 'Content-Encoding: gzip'
Documentation successfully uploaded for package candidate. You can now preview the result at
'https://hackage.haskell.org/package/HsYAML-aeson-0.2.0.1/candidate'. 
To upload non-candidate documentation, use 'cabal upload --publish'.

Maybe from running a similar command (with verbosity option -v) you can see how cabal is using curl and maybe you can succeed doing it similarly.

andreasabel avatar May 11 '22 21:05 andreasabel

Looks like the browser uses content-type=gzip and enconding=whatever instead.

dpwiz avatar May 12 '22 06:05 dpwiz

Could you fix your problem, @dpwiz ?

andreasabel avatar May 13 '22 06:05 andreasabel

No. Using browser still gives "unexpected content-type". And cabal upload -d still gives me 401.

dpwiz avatar May 13 '22 08:05 dpwiz

Please give some basic context: Version of

  • OS
  • curl
  • cabal
  • browser

What happens if you use curl in the way of https://github.com/haskell/hackage-server/issues/1070#issuecomment-1124320024 ?

andreasabel avatar May 13 '22 12:05 andreasabel

  • Linux (Ubuntu 22.04)
  • curl 7.81.0
  • cabal-install version 3.6.2.0
  • Chrome 100, Firefox 100

dpwiz avatar May 13 '22 16:05 dpwiz

Anyway, the details of my setup are irrelevant. The code states explicitly it will refuse compressed tarballs: https://github.com/haskell/hackage-server/blob/8ec64a1a30b6d53e4e86235c50e398ec803f50f1/src/Distribution/Server/Features/Documentation.hs#L312

(Yes, I can see curl instructions in there. I just like to have the browser upload form working, UX-wise.)

dpwiz avatar May 13 '22 16:05 dpwiz

I can confirm (in one instance) that the web interface for documentation upload rejected a .tar.gz but accepted the gunzipped version .tar. (Firefox 100)

I have never used the web form before, I always use cabal upload -d. Weird about the 401. Can you upload candidate packages (cabal upload)?

andreasabel avatar May 13 '22 19:05 andreasabel

Nah, cabal can't upload neither package nor docs.

dpwiz avatar May 15 '22 19:05 dpwiz

Well, a 401 is about missing authorization. Are your auth settings up-to-date in .cabal/config?

hackage-username: ...
hackage-password: ...

andreasabel avatar May 16 '22 05:05 andreasabel

There is a bug in its "curl" transport. I can upload docs for candidate, but it gets reset after publishing. And I can't upload docs to a non-candidate version even with "wget" transport.

dpwiz avatar May 16 '22 09:05 dpwiz