klaus icon indicating copy to clipboard operation
klaus copied to clipboard

Download tarballs are served from URLs ending with a slash

Open BonkaBonka opened this issue 7 years ago • 7 comments

Arch Linux's makepkg downloads a source tarball and ignores the content-disposition provided filename, instead seeming to parse the URL and taking the part after the final slash (and then appending .part to it). Thus generating errors that look like:

$ makepkg --syncdeps --noconfirm --sign
==> Making package: progcast 0.0.2-1 (Wed Nov  8 06:34:40 UTC 2017)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2412    0  2412    0     0   2412      0 --:--:-- --:--:-- --:--:--  168k
mv: '/home/aurbuild/arch_packaging/progcast/.part' and '/home/aurbuild/arch_packaging/progcast/.part' are the same file
==> Validating source files with md5sums...
     ... NOT FOUND
==> ERROR: One or more files did not pass the validity check!

The easiest tweak might just be to adjust the routing to remove the requirement for the terminal slash - I'm not sure.

BonkaBonka avatar Nov 13 '17 19:11 BonkaBonka

I found the bit in makepkg that uses the URL to set the output filename: https://git.archlinux.org/pacman.git/tree/lib/libalpm/dload.c#n52

BonkaBonka avatar Nov 13 '17 20:11 BonkaBonka

Suggestion: Add a route that ends with .tar.gz instead of /, make it the default tarball download route, and make the / route a permanent redirect to the new .tar.gz route. This solves the problem with pacman/alpm and it also provides for nice filenames if the download tool ignores Content-Disposition (curl, wget, possibly others).

jonashaag avatar Nov 13 '17 22:11 jonashaag

Actually, having read your gist in #204, I'd suggest going with the GitHub URL format for the new route instead, i.e.:

/<repo>/archive/<rev>.tar.gz

This allows for other formats than gzip tarballs in the future without awkward URLs.

jonashaag avatar Nov 13 '17 22:11 jonashaag

Actually, make it work exactly like cgit, as you suggested in #204. This is the best compromise.

/<repo>/archive/<repo>-<rev>.tar.gz

BUT shorten full SHAs to 7 characters, i.e. caddy-8f3846d.tar.gz instead of caddy-8f3846d48046bcd7678256b3ad2f70172d2ca1e8.tar.gz

And make the old tarball route NOT redirect, because curl needs -L to follow redirects, which is annoying.

jonashaag avatar Nov 13 '17 22:11 jonashaag

We can also do aside with the Content-Disposition header for the new route as suggested filename and URL filename will be identical.

jonashaag avatar Nov 13 '17 22:11 jonashaag

This would be awesome! Thank you very much for looking at all of these. :)

BonkaBonka avatar Nov 14 '17 17:11 BonkaBonka

Perhaps it would make sense to use the equivalent of "git describe" to name the file.

jelmer avatar Dec 12 '17 01:12 jelmer