RFC: offer `.tar.xz` downloads for extra byte savings

[petecooper]

As part of addressing https://github.com/textpattern/textpattern-toolbelt/issues/30, I have rolled up my sleeves on the Textpattern release toolbelt. The stdout is clearer, there's less noise, and it looks like this:

$ bash ./txp-gitdist.sh dev-archive-test /Users/pete/scratch /Users/pete/repos/textpattern/textpattern 


== Building textpattern-dev-archive-test.tar.gz in /Users/pete/scratch.
 - Built textpattern-dev-archive-test.tar.gz (1.98MB).


== Building textpattern-dev-archive-test.zip in /Users/pete/scratch.
 - Built textpattern-dev-archive-test.zip (2.31MB).


== Testing textpattern-dev-archive-test.tar.gz integrity...
 - textpattern-dev-archive-test.tar.gz passed `gzip -t` integrity test.
 - Calculating textpattern-dev-archive-test.tar.gz SHA256 checksum...
   b54023c250e6e9c51e3636a8e8f980d784f24eba4b21ea087adc67a13dcf100b
 - Checking textpattern-dev-archive-test.tar.gz checksum...
   textpattern-dev-archive-test.tar.gz: OK


== Testing textpattern-dev-archive-test.zip integrity...
   No errors detected in compressed data of textpattern-dev-archive-test.zip.
 - textpattern-dev-archive-test.zip passed `unzip -t` integrity test.
 - Calculating textpattern-dev-archive-test.zip SHA256 checksum...
   36fc36cb30f8676b77f4d843d0a0c1da5c96071a7c41395e786545363bae6982
 - Checking textpattern-dev-archive-test.zip checksum...
   textpattern-dev-archive-test.zip: OK


== Textpattern vdev-archive-test built in /Users/pete/scratch

Please note the difference in file size between the two archives. The .tar.gz is around 14% smaller than the .zip, even with the inclusion of the multisite scaffold.

I've duplicated the txp-dist.sh file as txp-dist-xz.sh here, and I've added the ingredients for making a Textpattern .tar.xz archive, and I've run it. The byte savings are even greater:

$ bash ./txp-gitdist-xz.sh dev-archive-test /Users/pete/scratch /Users/pete/repos/textpattern/textpattern 


== Building textpattern-dev-archive-test.tar.gz in /Users/pete/scratch.
 - Built textpattern-dev-archive-test.tar.gz (1.98MB).


== Building textpattern-dev-archive-test.tar.xz in /Users/pete/scratch.
 - Built textpattern-dev-archive-test.tar.xz (1.29MB).


== Building textpattern-dev-archive-test.zip in /Users/pete/scratch.
 - Built textpattern-dev-archive-test.zip (2.31MB).


== Testing textpattern-dev-archive-test.tar.gz integrity...
 - textpattern-dev-archive-test.tar.gz passed `gzip -t` integrity test.
 - Calculating textpattern-dev-archive-test.tar.gz SHA256 checksum...
   152ef7ffbeb66c20cd376f3804e7ebcab58fcbc932f5496303800f552fa9ed25
 - Checking textpattern-dev-archive-test.tar.gz checksum...
   textpattern-dev-archive-test.tar.gz: OK


== Testing textpattern-dev-archive-test.tar.xz integrity...
 - textpattern-dev-archive-test.tar.xz passed `xz -t` integrity test.
 - Calculating textpattern-dev-archive-test.tar.xz SHA256 checksum...
   6bfca485198315db5eb2929ed736187024272189417703d3946c1208b8055d0b
 - Checking textpattern-dev-archive-test.tar.xz checksum...
   textpattern-dev-archive-test.tar.xz: OK


== Testing textpattern-dev-archive-test.zip integrity...
   No errors detected in compressed data of textpattern-dev-archive-test.zip.
 - textpattern-dev-archive-test.zip passed `unzip -t` integrity test.
 - Calculating textpattern-dev-archive-test.zip SHA256 checksum...
   cc6d5399cc1a327c550be86a9cb4be88a49e81e0ebf717e9de346c215b545840
 - Checking textpattern-dev-archive-test.zip checksum...
   textpattern-dev-archive-test.zip: OK


== Textpattern vdev-archive-test built in /Users/pete/scratch

1.29MB vs 2.31MB. That's about 44% fewer bytes used than the .zip, and includes the multisite scaffold like the .tar.gz.

Can we consider offering .tar.xz downloads alongside the current .tar.gz and .zip? For bonus points: ideally on GitHub and our own site.

We'd need to change the markup on our site to include the new variant, and GitHub would be an additional archive upload with the associated checksum file.

Thank you.

Tagging https://github.com/textpattern/textpattern-toolbelt/issues/13 & https://github.com/textpattern/textpattern-toolbelt/issues/28 – third time's a charm!

[Bloke]

Oooh nice. Thanks for tidying that script up.

Nothing against .tar.xz. If it has widespread adoption, could we use it instead of .tar.gz? Or is it safer to include both?

[petecooper]

Go for both, .tar.gz is ubiquitous with Linux/UNIX types, and it's not going anywhere.

I get the feeling making .tar.xz archives will never get to widespread adoption (it's computationally much more expensive to create an archive), but the unpacking tools are widespread enough to make it worth it…and 44% saving is not to be sniffed at if we're watching our weight.

[Bloke]

Fine by me. We may need to invoke @philwareham if it impacts the .com site to offer all three formats.

[petecooper]

NTS: release .tar.xz for 4.9.0 beta 2 as a trial run.

[petecooper]

In theory, this is all now set up for 4.9.0 beta.2 - I will keep it open for now and close when I'm happy it's working as intended. There will be some mop up in docs & pophelp moving forward, but it can wait for the GA release.

[petecooper]

Toolbelt covers .tar.xz release process and website appears to be ready to receive .tar.xz, closing for now.