conditional compilation of embed-resource and tempfile to speed up the build?

[alsuren]

I noticed that cargo install cargo-quickinstall now takes ~6s on my mac. The bulk of this is compiling the build dependency embed-resource, which pulls in toml_edit and serde.

• [ ] This seems to have been added as part of https://github.com/cargo-bins/cargo-quickinstall/issues/254 to make 32bit windows happy. I manually commented this out and got rm target; cargo build --release down to ~3s. Can we make this conditional?
• [ ] Might be worth adding a cargo-deny check in CI to make sure we're never including serde as a dep or build dep by accident

The bulk of the rest of the time is tempfile + libc. Annoyingly there isn't a way to unzip a zipfile without making a temporary file. We are not using the fancy libc "tempfile that goes away once you close it" - we're using https://docs.rs/tempfile/latest/tempfile/struct.NamedTempFile.html which can be trivially implemented using https://doc.rust-lang.org/nightly/std/env/fn.temp_dir.html and a drop impl (might still need fastrand if we want to include randomness in the path, but maybe we don't need to bother if we're not expecting parallel invocations?)

• [ ] a) could we kill off tempfile by vendoring something similar to NamedTempFile?
• [ ] b) could we get cargo-binstall to also provide .tar files for all architectures, or switch to using the cargo-binstall tarballs from the cargo-quickinstall repo, which are already .tar files?
• [ ] c) could we conditionally skip tempfile on linux (doesn't help macos and windows though)?

Removing tempfile and embed-resource would bring the build down back down to ~2s.

... edit: it turns out guess_host_triple also pulls in libc, so I guess that's why we're not winning that much by killing off tempfile. This is https://github.com/cargo-bins/cargo-quickinstall/pull/136 . Killing this off too gets us down to 1.37s. Killing off the build.rs entirely gets us down to 1.17s, but I don't know if it's possible to conditionally enable a build.rs.

I reckon we should definitely do the embed-resource tweak, because it's a big win with no downsides. I don't know how far I would go with the other bits though.

[NobodyXu]

For embed-resource introduced in #262 , I don't think we could do that without breaking cross compilation.

If we do

[target.'cfg(windows)'.build-dependencies]

Then it would break cross compilation for windows on linux.

I think maybe we can help embed-resource to be more lightweight, by disabling default features of toml?

https://docs.rs/crate/toml/0.8.19/features

I think they probably only need parse but the default also enables display.

[NobodyXu]

I was thinking about killing it, by running embed-resource in CI and then commit the manifest into our github repository, however that doesn't seem possible.

https://docs.rs/embed-resource/latest/src/embed_resource/non_windows.rs.html

embed-resource actually compiles using cc-rs and link it.

[NobodyXu]

I think embed-resource, we should open a ticket in upstream to make toml optional and provide the information manifest through the build-script?

[alsuren]

Then it would break cross compilation for windows on linux.

We don't cross compile for windows though, do we? We use windows-latest for x86_64-pc-windows-msvc and aarch64-pc-windows-msvc so we should be fine?

[NobodyXu]

Well but our user could do that, if they decide to do cargo install cargo-quickinstall for windows, on a linux machine.

I'd imagine speeding up cargo-quickinstall is to mainly benefit users who are compuling cargo-quickinstall?

[alsuren]

Well but our user could do that, if they decide to do cargo install cargo-quickinstall for windows, on a linux machine.

I'm not completely sure what the use case is for this. I will have a go at making the cross compilation thing work anyway, and get back to you.

I'd imagine speeding up cargo-quickinstall is to mainly benefit users who are compuling cargo-quickinstall?

The original use case for cargo quickinstall was so that you can use rust-based tools in CI without having to compile them yourself or learn about how to use the github actions cache.

Github's ubuntu-latest runners come with a copy of rust already installed, so you could copy-paste:

cargo install cargo-quickinstall && cargo quickinstall ast-grep && ast-grep scan

and it will quickly run any ast-grep based linter rules you have. For linter rules like this, the run time is dominated by the installation step, and quick feedback on your PR is super-useful.

This is a bit of a contrived example (and most tools have snippets that you can copy-paste into github actions for quick installation anyway) but I still feel a bit of pride that you can be really lazy and not bother reading anyone's docs and still get quick CI runs using cargo-quickinstall exactly as you would use it locally.

[NobodyXu]

I'm not completely sure what the use case is for this. I will have a go at making the cross compilation thing work anyway, and get back to you.

Well I mean technically they can use cargo-install however they like.

It's true that cross-compilation isn't of much use since cargo-quickinstall is very fast to compile, and it's not how it is meant to be used.