Improve CLI compile/test experience for developers using Nix (or other non-rustup toolchains)

[nasadorian]

Problem

I'm a Rust developer coming to Solana from the EVM. My first experiences trying to use the CLI for SBF compile/test have been unsuccessful because I use the Nix package manager to configure my Rust development environments. After failing to get a simple contract compiled locally, I started to dig into the CLI code and see a lot of room to improve it with regard to parameterization and modularity.

For example, the cargo-build-sbf utility:

• Depends on platform-tools installation at a predefined or specified version
• Assumes tools are supposed to be in HOME, downloads and modifies user's ~/.cache/solana directory to install tools. Nix and other hermetic package managers do not allow installed tools to modify global directories, so compilation plainly fails after platform-tools download is invoked.
• Creates symlinks from tools into sdk/sbf directory
• Creates environment variables for LLVM utilities
• Assumes rustup exists and creates a solana toolchain, then invokes cargo +solana which only works with rustup
• Runs post-processing after compilation on artifacts

In my opinion this tool assumes a lot about the user's system and does too much under the hood. I realize this tool is intended for a majority of users that aren't concerned and want the tool to "just work" magically. But I feel that the code in its current state is too complicated and not composable.

Proposed Solution

• Refactor cargo-build-sbf to separate platform setup and installation from compilation
• Introduce more organized code for runtime dependencies (e.g. a struct to hold references to the tools)
• Add optional CLI arguments to specify locations or pin versions of tools to use
• Only use rustup if user has not specified a custom toolchain
• Consider configuration file (toml or nix) to make project builds reproducible
• Derive a standard flake.nix file to set up a clean Solana development environment that can readily run without incurring downloads or file system side effects (outside of target/).

I'm motivated to work on this myself, but would greatly appreciate input/pushback/comment from more experienced members of the community.

[Denommus]

My latest pull request for nixpkgs almost solves it, but I need help installing the platform-tools on the solana-validator package. I don't understand why, but when I run cargo build-bpf it fails with a permission error, but I've seen in the code that it's trying to install on $HOME/.cache/solana, so it shouldn't be a problem https://github.com/NixOS/nixpkgs/pull/332345