numtide
Autosourcing commands from `nix-index` & `nix-index-databse`
Is your feature request related to a problem? Please describe.
Treefmt is a great tool, but it doesn't take away the burdon of sourcing the used commands.
However, in UX categories, that would be in the ballpark of 50% of additional value (see Devenv, Fleek, Bob, etc).
treefmt-nix doesn't cut the deal, because it requires mental overhead, is not dead simple and interferes with my nix-lib project intent which is not commonly about boilerplate tooling or potentially not a nix project at all.
Describe the solution you'd like
- Setup and maintenance of CI should be as simple as:
- name: Install nix
uses: nixbuild/nix-quick-install-action@v22
- name: Check formatting
run: nix run nixpkgs#treefmt -- . --fail-on-change --no-cache
(non-pinning related issues with the tooling formatting are a rare "black-swan" in the horizons of this lifecycle, so we accept them in turn for the 50% additional UX)
- Set up and usage of
treefmtshould be as easy as:
- add
treefmt.toml - run
treefmt- for Nix users
which nix, this means feature-flagged (onwhich nix) profit - for no-nix users, this means no deterioration of their current expectations and a pretext to
sh <(curl -L https://nixos.org/nix/install), which might be a faster resolution of their problem at hand than any alternative course of action.
- for Nix users
Describe alternatives you've considered
treefmt-nix, manual management. It doesn't cut the deal, because it doesn't keep out of my way.
Additional context
I want to do a PR / contribution. Would it be accepted?
https://github.com/Mic92/nix-index-database/issues/44
In Github Actions a hypothetical and arbitrarily pinned nix/nixpkgs could speed up time-to-first-feedback some bits on style checks as it would already have nixpkgs downloaded` and - alas - eval cached.
Potentially, this implementation could attempt to fetch static builds where available to improve the situation further.
👎🏾 This sounds like a complex solution to a simple problem. each nix-index query takes up around 100ms + nix evaluation of the package to retrieve the attribute. nix-index-database might be not in sync with the nixpkgs channel of the user and it's not available for all architectures (only those built on hydra). Also naive usage without sorting might turn into funny results. I just picked the two random samples from the treefmt list and it resulted in this:
$ nix-locate bin/ocamlformat
ocamlformat_0_20_0.out 25,253,440 x /nix/store/hi3bf2l6byshkgh383v0z30gylv0xncz-ocaml4.14.1-ocamlformat-0.20.0/bin/ocamlformat
ocamlformat_0_22_4.out 24,745,384 x /nix/store/7h33cv3zxncci59xcjx6g5bwizccgmmi-ocaml4.14.1-ocamlformat-0.22.4/bin/ocamlformat
ocamlformat_0_22_4.out 24,812,896 x /nix/store/7h33cv3zxncci59xcjx6g5bwizccgmmi-ocaml4.14.1-ocamlformat-0.22.4/bin/ocamlformat-rpc
ocamlformat_0_22_4.out 11,462,152 x /nix/store/7h33cv3zxncci59xcjx6g5bwizccgmmi-ocaml4.14.1-ocamlformat-0.22.4/bin/ocamlformat.parser_recovery.test_driver
ocamlformat_0_22_4.out 935,576 x /nix/store/7h33cv3zxncci59xcjx6g5bwizccgmmi-ocaml4.14.1-ocamlformat-0.22.4/bin/ocamlformat.parser_recovery.test_gen
ocamlformat_0_23_0.out 24,725,664 x /nix/store/8hxnk172dvcr1vpahhavf2l3r2gp2iyx-ocaml4.14.1-ocamlformat-0.23.0/bin/ocamlformat
ocamlformat_0_23_0.out 24,789,080 x /nix/store/8hxnk172dvcr1vpahhavf2l3r2gp2iyx-ocaml4.14.1-ocamlformat-0.23.0/bin/ocamlformat-rpc
ocamlformat_0_23_0.out 11,480,000 x /nix/store/8hxnk172dvcr1vpahhavf2l3r2gp2iyx-ocaml4.14.1-ocamlformat-0.23.0/bin/ocamlformat.parser_recovery.test_driver
ocamlformat_0_23_0.out 935,576 x /nix/store/8hxnk172dvcr1vpahhavf2l3r2gp2iyx-ocaml4.14.1-ocamlformat-0.23.0/bin/ocamlformat.parser_recovery.test_gen
ocamlformat.out 24,735,976 x /nix/store/gpv0cydm7bv8j7yfz94g7zqchayzpy8l-ocaml4.14.1-ocamlformat-0.24.1/bin/ocamlformat
ocamlformat.out 24,799,392 x /nix/store/gpv0cydm7bv8j7yfz94g7zqchayzpy8l-ocaml4.14.1-ocamlformat-0.24.1/bin/ocamlformat-rpc
ocamlformat.out 11,480,000 x /nix/store/gpv0cydm7bv8j7yfz94g7zqchayzpy8l-ocaml4.14.1-ocamlformat-0.24.1/bin/ocamlformat.parser_recovery.test_driver
ocamlformat.out 935,576 x /nix/store/gpv0cydm7bv8j7yfz94g7zqchayzpy8l-ocaml4.14.1-ocamlformat-0.24.1/bin/ocamlformat.parser_recovery.test_gen
ocamlformat_0_19_0.out 23,296,544 x /nix/store/3nsvhijvxll5bf2q27nx4yn74acdfm6i-ocaml4.14.1-ocamlformat-0.19.0/bin/ocamlformat
ocamlformat_0_20_1.out 25,266,024 x /nix/store/hlxkf658gb42y4m7i7dc5m62xsplnm72-ocaml4.14.1-ocamlformat-0.20.1/bin/ocamlformat
ocamlformat_0_21_0.out 25,338,400 x /nix/store/0lfla9xg9qnxn8g66zpgbr50f535kys9-ocaml4.14.1-ocamlformat-0.21.0/bin/ocamlformat
$ nix-locate bin/hlint
ihaskell.out 293 x /nix/store/5q85ynqif5i9if3fw7ci6gw241iar8x4-ihaskell-with-packages/bin/hlint
(ihaskell.out) 0 s /nix/store/6pwqjhsfrzaz3krr5k49954s5628076d-ghc-9.2.7-with-packages/bin/hlint
hlint.out 22,306,464 x /nix/store/26s0yailyl0284ky95mz4j39z2g0vh6b-hlint-3.4.1/bin/hlint
haskellPackages.hlint.out 22,306,464 x /nix/store/aifqw524819445nzsxi2grn1l7ynv8kc-hlint-3.4.1/bin/hlint
haskellPackages.hlint-test.out 934,400 x /nix/store/1smkch8jfcd51cl31lxr7sxcr0il57x8-hlint-test-0.1.0.0/bin/hlint-test
Every by default the tool would have chosen ocamlformat_0_20_0 and ihaskell, which would be odd choices.
I appreciate we may find ourselves in a design process in search of the best design of a solution to the problem statement (UX - centered).
I'm not sure, though, how you'd address the problem statement? Maybe we already got carried away on the implementation details?
How do other great UX-optimized tools provide good enough heuristics?
the burdon of sourcing the used commands.
Have you tried JRMurr/direnv-nix-action which bundles cachix/install-nix-action and direnv?
So assuming you have a use_flake in your .envrc you can:
- name: "Setup env"
uses: JRMurr/direnv-nix-action@v2
- run: treefmt
That is a prohibitively solution.
Let me explain:
- The key CI metric is cycle time
- On chores like fmt checks, it typically is Time-To-First feedback
- Now suppose you lock your dependencies (instead of using a docker image that pins
nixpkgsand even holds an eval cache), you need to:- download nixpkgs (~2s)
- eval the cache (~13s)
- download typically non-static binaries; that isn't efficient at all
- Now, you "just" use
direnv, and voila, you've got your Time-To-First feedback coupled with the inital (uncached) loading of the env - Ad an expensive IFD such as haskell.nix and you're waiting long long minutes.
- At this point, your Nix based solution simply doesn't meet the productivity requirement of your team.
Instead, this proposal holds:
- Pin by environment, which:
- leverages baked in
nixpkgsand its eval cache (-10s)
- leverages baked in
- Use static build
- matches against static builds (which are potentially instant to fetch) (-45s)
- JIT and demand-oriented sourcing
- only sources JIT what's needed (-30min - depending on the env)
I think it's an obvious design choice, when you we want to increase the user acceptance for treefmt (so that we can use the tools we love, when we're not king).
There will be controversy on the heuristics and my preferred option would be to have a community maintained matcher tool for CLI purposes so that we can have the discussion once and not every new tool has to roll with their own solution.
As @Mic92 suggested the initially considered backend might not meet the requirements.
As a Nix user, I expect to be able to pin the versions of my software, and that treefmt would use those same versions. I don't really see how this can be achieved without evaluating the nix code?
Let's say that there is a Nix evaluation snapshot that is possible, it seems to me that this is a generic tool that can be solved outside of treefmt (it would be cool if it existed!). Something akin to Nix binary cache but for evaluation outputs.
One part of the speed issue is that devshells tend to include a lot of unrelated tools that get downloaded. This is why treefmt-nix exists, then you can nix fmt and it will only download the related tools. I'm not sure why editing a treefmt.nix is more complicated than editing a treefmt.toml?
In the same design space, it would also be nice if devshells supported lazy loading.
Another part is that CI runners don't maintain a local /nix/store cache between runs. For this, we have srvos github actions runners that you can self host, or use Hercules CI or Garnix.
As a Nix user, I expect to be able to pin the versions of my software, and that treefmt would use those same versions. I don't really see how this can be achieved without evaluating the nix code?
As everyone else, ... ... multilevel UX problems may be blocking for adoption. Including one of them being the verbosity of Nix.
That person is not me, and I'm assuming the role of acting attorney for people I've interacted with around this topic, but who stay below the threshold to care (about treefmt).
Let's say that there is a Nix evaluation snapshot that is possible, it seems to me that this is a generic tool that can be solved outside of treefmt.
In my mind the simplest potential implementation is nixos/nix:latest-with-eval-cache (docker). Though this, indeed, is orthogonal.
What treefmt can offer is to apply a heuristic command resolution as opt-in service to the non-Nix user persona.
Very oft they don't share at all the same set of values and goals with the Nix persona. That must always be kept in mind.
Heuristic command resolutio eventually may be provided as a Nix ecosystem capability in the form of a library. Treefmt could adopt it, and so could other tools (bob, fleek, etc) converge to shared maintenance.
How thet ultimately choose to feature flag such capability is completely secondary, though feature flagging on the presence of nix seems innocuous and handy (when dealing with non-nix config data).
Another part is that CI runners don't maintain a local /nix/store cache
Outside of Nix, static builds have gained popularity to minimize the impact of missing caches.
This would be a workable solution, here, too.
To clarify my stance, I do acknowledge, than within the Nix Ecosystem, we have different solutions to those problems.
But that isn't really the point here.
We already establish above that goals and values don't necessarily align between Nix and non-Nix user persona.
The logical consequence of this assessment is that giving the non-Nix persona a choice between "making those solution a precondition" or "suffering UX" is a hinderance to (viral) adoption of treefmt.
If treefmt can be considered an entry level drug (I think it can), it may not realize its full potential to help drive adoption of Nix itself.
A draft component-diagram-ish of a mapper's principle. Meant to be design context, not a proposed solution.