stylix icon indicating copy to clipboard operation
stylix copied to clipboard

flake: partition dev inputs

Open 0xda157 opened this issue 7 months ago • 3 comments

follow up to #1208 cc @trueNAHO @MattSturgeon

Things done

Notify maintainers

0xda157 avatar May 17 '25 00:05 0xda157

currently marked as a draft because the testbed logic needs to be refactored to enable flake-parts to see what testbeds exist as mentioned in https://github.com/danth/stylix/pull/1208#issuecomment-2878394791. not entirely sure how to go about that, would probably be best to do in a separate pr.

0xda157 avatar May 17 '25 00:05 0xda157

currently marked as a draft because the testbed logic needs to be refactored to enable flake-parts to see what testbeds exist as mentioned in https://github.com/danth/stylix/pull/1208#issuecomment-2878394791. not entirely sure how to go about that, would probably be best to do in a separate pr.

My vague idea is to split testbed.nix's autoload into two stages.

Stage 1 collects the testbed modules and forms them into an attrset of {name → module}. This bit is needed in the top-level flake configuration.

Stage 2 maps those modules into nixos VM systems. This is done in the dev partition.

This would probably need to be done in separate files, as otherwise it'll be difficult to call the testbed.nix file from the flake, without supplying inputs needed for Stage 2.

MattSturgeon avatar May 17 '25 03:05 MattSturgeon

My vague idea is to split testbed.nix's autoload into two stages.

Stage 1 collects the testbed modules and forms them into an attrset of {name → module}. This bit is needed in the top-level flake configuration.

Stage 2 maps those modules into nixos VM systems. This is done in the dev partition.

This would probably need to be done in separate files, as otherwise it'll be difficult to call the testbed.nix file from the flake, without supplying inputs needed for Stage 2.

I think avoiding a major refactor of the testbeds would be a bad idea just because there's a lot of other prs editing it them, hopefully we can get them merged soon.

0xda157 avatar May 20 '25 02:05 0xda157

I believe all blockers are now resolved, so things should work once rebased :crossed_fingers:


I'd also like to refactor things so that we have an internal flake-parts option for the autoloaded testbed modules set; this could be re-used by both partitions.

Although DRY-er it doesn't improve efficiency, because the partition will always re-evaluate the entire flake module configuration anyway. So importing directly is fine too.

This idea to put the testbed modules in a common flake-parts option shouldn't block this PR, it can be done as part of this PR, before this PR, after this PR, or not at all.

MattSturgeon avatar Jun 26 '25 10:06 MattSturgeon

I believe all blockers are now resolved, so things should work once rebased 🤞

seems to be having inf-rec issues

I'd also like to refactor things so that we have an internal flake-parts option for the autoloaded testbed modules set; this could be re-used by both partitions. Although DRY-er it doesn't improve efficiency, because the partition will always re-evaluate the entire flake module configuration anyway. So importing directly is fine too.

This idea to put the testbed modules in a common flake-parts option shouldn't block this PR, it can be done as part of this PR, before this PR, after this PR, or not at all.

would be best to do this in a future pr

0xda157 avatar Jun 26 '25 18:06 0xda157

seems to be having inf-rec issues

Looks like it's the module system, while evaluating the flake option? Or possibly while evaluating module args?

Can investigate locally a bit later.

would be best to do this in a future pr

My thought process was that having it sooner may simplify this PR, but that I didn't want to block this PR if I didn't get around to it.

MattSturgeon avatar Jun 26 '25 18:06 MattSturgeon

seems to be having inf-rec issues

Looks like it's the module system, while evaluating the flake option? Or possibly while evaluating module args?

Can investigate locally a bit later.

would be best to do this in a future pr

My thought process was that having it sooner may simplify this PR, but that I didn't want to block this PR if I didn't get around to it.

it looks like it's coming from the eval of isLinux for the testbeds here https://github.com/nix-community/stylix/blob/4b7df6434b7455aeee98ee538fa395590beafad3/flake/propagted-packages.nix#L17

0xda157 avatar Jun 26 '25 18:06 0xda157

everything seems to be working now :tada:

0xda157 avatar Jun 26 '25 19:06 0xda157

[!tip] You can test whether evaluating a specific flake output will evaluate the flake partition with this patch:

diff --git a/flake/default.nix b/flake/default.nix
index bd64e95..f07fd08 100644
--- a/flake/default.nix
+++ b/flake/default.nix
@@ -9,7 +9,7 @@
   ];
 
   partitions.dev = {
-    module = ./dev;
+    module = builtins.trace "evaluating dev partition" ./dev;
     extraInputsFlake = ./dev;
   };
 

E.g.

$ nix eval .#doc
«derivation /nix/store/sxnsnb5nnk1ayf3pz59aynh6ihy294hc-stylix-book.drv»

$ nix eval .#testbed:alacritty:cursorless
trace: evaluating dev partition
«derivation /nix/store/88nzq7nhqagabx22d5z9d9q5qf6yng9j-testbed-alacritty-cursorless.drv»

$ nix eval .#checks.x86_64-linux.doc
trace: evaluating dev partition
«derivation /nix/store/sxnsnb5nnk1ayf3pz59aynh6ihy294hc-stylix-book.drv»

It is important that trace: evaluating dev partition does not show up for user-facing outputs, otherwise we defeat the point of having a partition in the first place.

MattSturgeon avatar Jun 26 '25 20:06 MattSturgeon

@danth @trueNAHO merge conflict are getting annoying to deal with and this is now blocking https://github.com/nix-community/stylix/pull/1579, ideally we could get this merged soon

0xda157 avatar Jul 03 '25 18:07 0xda157

[PATCH 7/9] flake: dev: nixpkgs-rev: rewrite Bash script in pure Nix

My issue with this is the check is done at eval time, and throws if it fails.

If you want to evaluate and build all checks, this will fail the eval, so you can't see which other checks would also fail (unless you eval them individually, like nix-eval-jobs does).

If you do something like nix flake show, failing eval would not be ideal. This is problematic if someone did a nix flake show with overridden inputs, although that is an unlikely scenario.

Either way, a nix flake check with overridden inputs would fail (as expected), but by doing the assertion in the buildscript, you allow building with --keep-going to check the other tests.

If you don't care about any of this, and don't mind doing assertions when evaluating the check instead of when actually running (building) the check, then this isn't a big deal.

MattSturgeon avatar Jul 04 '25 17:07 MattSturgeon

Just to clarify, does this change have no impact on nix usage for Stylix contributors and no impact for consumers of the Stylix flake (except that dev inputs are no longer required to be disabled, but nix already provides a good warning message about this "breaking" change)?

nix flake lock or nix flake update will automatically remove transitive nodes that are no longer needed from the consumer's lockfile.

If the consumer's flake is configuring an input that no longer exists (e.g. with follows), updating the lockfile will either warn or error; stating that the referenced input doesn't exist. I can't remember the exact message or whether it is a warning or an error, but this should be easy to test if needed.

MattSturgeon avatar Jul 05 '25 16:07 MattSturgeon

If the consumer's flake is configuring an input that no longer exists (e.g. with follows), updating the lockfile will either warn or error; stating that the referenced input doesn't exist. I can't remember the exact message or whether it is a warning or an error, but this should be easy to test if needed.

It's a warning, for example

warning: input 'nur' has an override for a non-existent input 'systems'

0xda157 avatar Jul 05 '25 20:07 0xda157

Q: In flake/modules.nix you inherit stylix's inputs into the various modules, which defines the stylix.inputs option.

Since the modules come from the public flake, those inputs will not include the dev inputs.

Technically that's a breaking change, but I assume none of the inputs moved to the dev partition were intended to be accessed via the stylix.inputs option? And I assume stylix doesn't use them in any modules?

Any users that are reading the dev inputs from the stylix.inputs option would get nix's usual "attribute 'abc' not found, did you mean 'xyz'" error.

MattSturgeon avatar Jul 05 '25 20:07 MattSturgeon

Q: In flake/modules.nix you inherit stylix's inputs into the various modules, which defines the stylix.inputs option.

Since the modules come from the public flake, those inputs will not include the dev inputs.

Technically that's a breaking change, but I assume none of the inputs moved to the dev partition were intended to be accessed via the stylix.inputs option? And I assume stylix doesn't use them in any modules?

Any users that are reading the dev inputs from the stylix.inputs option would get nix's usual "attribute 'abc' not found, did you mean 'xyz'" error.

The dev inputs shouldn't be included in config.stylix.inputs, it's meant to just be deps of the public modules (as I understand)

0xda157 avatar Jul 06 '25 00:07 0xda157

The dev inputs shouldn't be included in config.stylix.inputs, it's meant to just be deps of the public modules (as I understand)

Correct. I didn't mean to imply otherwise.

I was a) highlighting the "subtly breaking" change and b) double checking that none of the inputs moved to the dev partition should actually be kept in the public flake.lock

MattSturgeon avatar Jul 06 '25 00:07 MattSturgeon

Backport failed for release-25.05, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-25.05
git worktree add -d .worktree/backport-1289-to-release-25.05 origin/release-25.05
cd .worktree/backport-1289-to-release-25.05
git switch --create backport-1289-to-release-25.05
git cherry-pick -x a5c1532ab8bf9566331f83be91517e02132f3d99

stylix-automation[bot] avatar Jul 06 '25 21:07 stylix-automation[bot]

I will take care of the manual backport

0xda157 avatar Jul 06 '25 21:07 0xda157