Customize binary output location

[Blacksmoke16]

Currently shards build always outputs the binaries in the bin directory. Being able to customize this would make things a lot more flexible, especially when shards is being used within another package manager.

For example I'm working on a v2 Crystal snap plugin. It expects that the built binaries are available at a specific location. At the moment I'm doing 'cp -r ./bin "${SNAPCRAFT_PART_INSTALL}"/bin', but that feels kinda meh. I've been referencing Go and Rust implementations and they have GOBIN and --root respectively.

@straight-shoota also suggested that we could utilize the -o flag with some special semantics on if it's a directory or not.

So guess we just need to figure out what approach we want:

1. ENV var
2. A dedicated option
3. Reuse -o with some special semantics

Originally posted by @Blacksmoke16 in https://github.com/crystal-lang/shards/issues/179#issuecomment-872430987

[straight-shoota]

My answers to such feature requests might repeat themselves, but there's another option:

4. Use an actual build system (e.g. make myshard O="${SNAPCRAFT_PART_INSTALL}"/bin)

[Blacksmoke16]

@straight-shoota Wouldn't this require every shard to use a Makefile? Or would at least require snapcraft to define a generic one; which is a bit overkill versus just doing cp.

[straight-shoota]

Yes, I think every non-trivial shard would benefit from using a build system for such tasks. Ideally make, because that has most widespread adoption.

[asterite]

Just note that my idea with Crystal was to be able to run crystal to compile a program. I didn't read this thread, but I hope that doesn't change!

[asterite]

I don't understand. If shards build already produces a binary, why not make that binary location customizable? It seems trivial to do. Compiling a C project and so on seems like a task for a build tool, but configuring an output directory? I don't think so.

[straight-shoota]

It's not super trivial. shards build arguments are very simply: any argument starting with - is forwarded to crystal build, any other argument is treated as target name. If we want to customize the behaviour of shards build itself via command line arguments, we need to distinct local and forward arguments. That adds more complexity to shards for a task that is IMO outside its principal use case.

[asterite]

I doubt the output location needs to be changed per run, so it could be specified in the yaml file

[straight-shoota]

I can't make any sense of that. What would be the use case?

The OP mentions the use case of putting build artifacts in a destination directory for a snapcraft distribution. Such a path (in the example $SNAPCRAFT_PART_INSTALL/bin) is very specific to the local environment and install intention. If you don't want to package for snapcraft, it's totally reasonable to build to the local bin/ directory as per default, using the same shard.yml. If there is to be an option for a custom output path, that must be configurable at runtime.

In my opinion however, this is a typical use case for make install. The binary gets built locally using shards build (which can be invoked by make build) and then make install takes care of installing at the desired destination. This is a very commonly used and much more flexible solution. Using a custom output path for shards build is very limited because it only applies to Crystal built binaries. It cannot take care of any other artifacts such as (binary) libraries, configuration, manpages, license etc.

[gofenix]

https://doc.rust-lang.org/cargo/commands/cargo-install.html

we can add install command like cargo install. It will easily build some executable files

[luislavena]

👋🏽 thinking on future developer ergonomics (DX) for a bit and given shards build as a thin wrapper of crystal build, wouldn't make sense to have a way to indicate the root directory of that build? Right now, you can indicate where crystal build will output the generated binary.

Imagine in the future Crystal allowed and worked in a way that crystal build --target provided an out-of-the-box experience for cross-compilation and linking on the same system (host). The biggest difference and disadvantage of shards build is that when building one or multiple targets, you cannot indicate where to place them.

Say:

$ shards build myapp --root build/host/bin

$ shards build myapp --target x86_64-linux-musl --static --root build/x86_64-musl/bin

$ shards build myapp --target aarch64-linux-musl --static --root build/arm64-musl/bin

Heck, why not:

$ shards build myapp --target x86_64-windows-msvc --root build/x86_64-windows/bin

🌈 😊

Pushing this to another build tool (make) just adds another dependency on a chain, which I think contradicts the idea of a thin wrapper around crystal build. Having that path hardcoded inside shard.yml will also limit certain arguments that are proxied to Crystal compiler, like target, forcing you to build and move files around before the next build.

Of course, this assumes you could cross-compile to these targets without issues (but that is a dream for another issue in another repository) 😉

Talking about packaging itself (Snap, apk, deb, etc) seems a different story, in which case neither shards or Crystal could deal with any package-specific nuances. In that case, you as developer take care of placing all the artifacts you want to package.

Cheers, ❤️ ❤️ ❤️

[straight-shoota]

@luislavena You can already do everything you mentioned.

$ shards build myapp --cross-compile --target=aarch64-linux-musl -obuild/arm64-musl/bin/myapp
Dependencies are satisfied
Building: myapp
cc build/arm64-musl/bin/myapp.o -o build/arm64-musl/bin/myapp  -rdynamic -L/home/linuxbrew/.linuxbrew/lib -lpcre -lm -lgc -lpthread -L/home/linuxbrew/.linuxbrew/Cellar/libevent/2.1.12/lib -levent -lrt

[luislavena]

Right @straight-shoota, but you cannot do that with all targets automatically, you need to do it target by target.

Apologies for not showing that in my example, copy and pasta failure 🤦🏼‍♂️