Add some support for filtering `src` or at least highlight it in the docs and examples.

[dpc]

After I realized why my projects rebuilds completely any time I touch any file it became apparent that src = ./.; is never a thing that one should use in practice. "src = ./.; considered harmful".

I see that you've opened #78 as I'm typing in this text, but I don't think it is enough.

Now that I discovered this fundamental improvement, I'll have to go and update all the projects I already started using crane in, which means copying bunch of boilerplate to filter src.

It seems to me that crane should come with a built-in and advertised function(s) to filter the src - possibly something like this and maybe also some specialized versions ("all Cargo files", "all cargo files + rs files"). At very least it would save on the boilerplate in every project. Also - probably the examples should just use such filtering everywhere. I don't think hoping for people to read FAQ and connect the dots is an optimal strategy. 99% of Rust users do not know Nix, yet they might want to try it out, or will want to contribute to projects that do use crane. They might not even be fully aware of how cargo uses target - they just want to build stuff and get a nice, fast reproducible build.

I remember that you mentioned somewhere that you'd like to have better automated filter, but it will be complex, and I'm not even sure if it's possible in general case (stuff like build.rs can potentially do anything inside, and how is crane going to figure it out?). Eg. when running tests, I need to enable some extra files anyway. So it seems to me that crane will have to support manual filtering for a long while - might as well make it first class.

[ipetkov]

Thanks for opening an issue for this!

I had originally used src = ./. in a lot of the flake examples because I wanted to make it easy for new comers to get started without overloading them with a bunch of concepts like source filtering on top of trying to illustrate the crane APIs.

That said I wholeheartedly agree that using src = ./. is almost always a bad idea since even just updating the formatting in flake.nix would cause a rebuild...

I do think having some utility filters would be useful and we should highlight them in the examples. At first I didn't want to make the examples super opinionated on what files should or shouldn't be kept (esp. if someone has a complex build which needs a lot of code generation or something). That said, I still think having something which keeps all *.rs files is a great option and anyone who needs more control can substitute their own filter!

Perhaps as long as we don't build it directly into cargoBuild it won't cause too much issue for power users, but we can iterate on the best API design even if the filtration is done separately for now

[dpc]

Perhaps as long as we don't build it directly into cargoBuild it won't cause too much issue for power users

Yeah. I think:

src = craneLib.standardRustSrcFilter ./.;

and

src = craneLib.standardRustSrcDepsOnlyFilter ./.;

would be good enough. Anyone that needs something more custom can just not use them.

This is barely longer than src = ./.;, raises the awareness of need for source filtering and is a great starting point that will work for 99% of the projects with a simple copy&paste.

I wanted to make it easy for new comers to get started without overloading them with a bunch of concepts like source filtering on top of trying to illustrate the crane APIs.

I'm afraid that at least in my case it backfired. And I'm already somewhat familiar with Nix. The hard part is to build a good mental model of what is going on under the hood, and then figure out how exactly to compose it into well working whole for my projects. How do I organize caching? Multiple packages? Reuse/share builds? Run tests? Checks lints? How to make it fast?

Only today I feel like I finally have all the pieces figured out.

To go a bit off-topic:

I believe that crane could become a go-to solution for CI in Rust. Builds taking forever in Rust is a common issue and complain. However as is right now crane does not do a great job at advertising the end-result advantages.

99% of Ruts users are unfamiliar with Nix. Rust already has a steep learning curve. Now looking at crane they would need to dive into yet another hard to understand thing. Why would they, if all they can understand is that it is just building Rust project? README says "source fetching" and "incremental" and "composable" which are: obvious + 2x very abstract. They can already do that with standard Rust github action templates.

Until I plugged in cachix, I didn't even realize how great the resulting CI is. Even tests don't have to run twice if they've ran successfully previously. That's super-cool and will save us a lot of time.

IMO, crane should have a sister repository (crane-demo) with a fake workspace-based project; with cachix, CI pipeline workflow, all the best ideas and practices and a lot of comments, and use that to showcase the posibilities. Kind of like what rustshop is, but rustshop is trying to cram a lot of things into one project so might confuse and scare people.

BTW. I planning to write some documentation for this crane-based CI I just implemented for fedimint, and hopefully it will useful enough to post on r/rust and showcase crane and attract some attention.