nixpkgs icon indicating copy to clipboard operation
nixpkgs copied to clipboard

nixos/crowdsec: use sensible defaults

Open TornaxO7 opened this issue 3 months ago • 30 comments

This PR should just refactor the crowdsec module.

Alright, so after looking a bit through the code, I've got the following suggestion:

  • I'd like to merge services.crowdsec.localConfig with services.crowdsec.settings into one config attribute or something similar because in my opinion it's really confusing to have to "config" attributes.
  • ~~The generated config file should be ~~written~~ symlinked to the default path (/etc/crowdsec/config.yaml) which should fix https://github.com/NixOS/nixpkgs/issues/445337~~
  • I'd remove the services.crowdsec.settings.capi and services.crowdsec.settings.lapi since the user can set them through services.crowdsec.settings.general.

Feel free to share your thoughts on my suggestions!

Things done

  • Built on platform:
    • [ ] x86_64-linux
    • [ ] aarch64-linux
    • [ ] x86_64-darwin
    • [ ] aarch64-darwin
  • Tested, as applicable:
  • [ ] Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • [ ] Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • [ ] Package update: when the change is major or breaking.
  • NixOS Release Notes
    • [ ] Module addition: when adding a new NixOS module.
    • [ ] Module update: when the change is significant.
  • [ ] Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a :+1: reaction to pull requests you find important.

TornaxO7 avatar Sep 26 '25 10:09 TornaxO7

Alright, I added my changes for now.

The only option which I currently have is services.crowdsec.enable = true;. However, I'm getting the following error after restarting the service:

Screenshot_2025-09-26_22-47-36

and I'm unsure what the cause may be.

Another big change is, that there's now only services.crowdsec.settings which combines the stuff from services.crowdsec.localConfig and services.crowdsec.settings.

@nicomem may I ask if you can try out my branch if it fixes https://github.com/NixOS/nixpkgs/issues/445342 for you? Do you get the same error as I do?

TornaxO7 avatar Sep 26 '25 20:09 TornaxO7

Thanks, I'll try your branch and get back to you on my results.

I also got this error when I was trying to make it work on my machine but I don't remember exactly what fixed it.

nicomem avatar Sep 27 '25 07:09 nicomem

After applying https://github.com/NixOS/nixpkgs/pull/446307#discussion_r2384161176 and https://github.com/NixOS/nixpkgs/pull/446307#discussion_r2384163782, the crowdsec service runs without failing.

However, it does not work correctly as if I call cscli parsers list, I get the following:

WARNING Ignoring file /etc/crowdsec/parsers/s00-raw/syslog-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s00-raw/crowdsecurity/syslog-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s01-parse/authelia-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s01-parse/LePresidente/authelia-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s01-parse/caddy-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s01-parse/crowdsecurity/caddy-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s01-parse/gitea-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s01-parse/LePresidente/gitea-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s01-parse/sshd-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s01-parse/crowdsecurity/sshd-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s02-enrich/crowdsecurity/dateparse-enrich.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s02-enrich/crowdsecurity/geoip-enrich.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s02-enrich/http-logs.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s02-enrich/crowdsecurity/http-logs.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/parsers/s02-enrich/public-dns-allowlist.yaml: lstat /var/lib/crowdsec/state/hub/parsers/s02-enrich/crowdsecurity/public-dns-allowlist.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/postoverflows/s00-enrich/rdns.yaml: lstat /var/lib/crowdsec/state/hub/postoverflows/s00-enrich/crowdsecurity/rdns.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/postoverflows/s01-whitelist/cdn-whitelist.yaml: lstat /var/lib/crowdsec/state/hub/postoverflows/s01-whitelist/crowdsecurity/cdn-whitelist.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/postoverflows/s01-whitelist/seo-bots-whitelist.yaml: lstat /var/lib/crowdsec/state/hub/postoverflows/s01-whitelist/crowdsecurity/seo-bots-whitelist.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2017-9841.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2017-9841.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2019-18935.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2019-18935.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-26134.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-26134.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-35914.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-35914.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-37042.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-37042.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-40684.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-40684.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-41082.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-41082.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-41697.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-41697.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-42889.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-42889.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-44877.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-44877.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2022-46169.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2022-46169.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2023-22515.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2023-22515.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2023-22518.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2023-22518.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2023-49103.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2023-49103.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2024-0012.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2024-0012.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2024-38475.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2024-38475.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/CVE-2024-9474.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/CVE-2024-9474.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/apache_log4j2_cve-2021-44228.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/apache_log4j2_cve-2021-44228.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/authelia-bf.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/LePresidente/authelia-bf.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/f5-big-ip-cve-2020-5902.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/f5-big-ip-cve-2020-5902.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/fortinet-cve-2018-13379.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/fortinet-cve-2018-13379.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/gitea-bf.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/LePresidente/gitea-bf.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/grafana-cve-2021-43798.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/grafana-cve-2021-43798.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-admin-interface-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-admin-interface-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-backdoors-attempts.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-backdoors-attempts.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-bad-user-agent.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-bad-user-agent.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-crawl-non_statics.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-crawl-non_statics.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-cve-2021-41773.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-cve-2021-41773.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-cve-2021-42013.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-cve-2021-42013.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-cve-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-cve-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-generic-bf.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-generic-bf.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-generic-test.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-generic-test.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-open-proxy.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-open-proxy.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-path-traversal-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-path-traversal-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-sap-interface-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-sap-interface-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-sensitive-files.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-sensitive-files.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-sqli-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-sqli-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-w00tw00t.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/ltsich/http-w00tw00t.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-wordpress-scan.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-wordpress-scan.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/http-xss-probing.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/http-xss-probing.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/jira_cve-2021-26086.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/jira_cve-2021-26086.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/netgear_rce.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/netgear_rce.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/pulse-secure-sslvpn-cve-2019-11510.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/pulse-secure-sslvpn-cve-2019-11510.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/spring4shell_cve-2022-22965.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/spring4shell_cve-2022-22965.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/ssh-bf.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/ssh-bf.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/ssh-cve-2024-6387.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/ssh-cve-2024-6387.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/ssh-generic-test.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/ssh-generic-test.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/ssh-refused-conn.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/ssh-refused-conn.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/ssh-slow-bf.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/ssh-slow-bf.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/thinkphp-cve-2018-20062.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/thinkphp-cve-2018-20062.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/vmware-cve-2022-22954.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/vmware-cve-2022-22954.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/scenarios/vmware-vcenter-vmsa-2021-0027.yaml: lstat /var/lib/crowdsec/state/hub/scenarios/crowdsecurity/vmware-vcenter-vmsa-2021-0027.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/contexts/bf_base.yaml: lstat /var/lib/crowdsec/state/hub/contexts/crowdsecurity/bf_base.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/contexts/http_base.yaml: lstat /var/lib/crowdsec/state/hub/contexts/crowdsecurity/http_base.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/authelia.yml: lstat /var/lib/crowdsec/state/hub/collections/LePresidente/authelia.yml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/base-http-scenarios.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/base-http-scenarios.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/caddy.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/caddy.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/gitea.yml: lstat /var/lib/crowdsec/state/hub/collections/LePresidente/gitea.yml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/http-cve.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/http-cve.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/linux.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/linux.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/sshd.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/sshd.yaml: no such file or directory 
WARNING Ignoring file /etc/crowdsec/collections/whitelist-good-actors.yaml: lstat /var/lib/crowdsec/state/hub/collections/crowdsecurity/whitelist-good-actors.yaml: no such file or directory 
──────────────────────────────────────
 PARSERS                              
──────────────────────────────────────
 Name  📦 Status  Version  Local Path 
──────────────────────────────────────
──────────────────────────────────────

nicomem avatar Sep 27 '25 13:09 nicomem

From what I have seen, the idiomatic way to make /var/lib/crowdsec and /etc/crowdsec accessible to the service, is to use the systemd service options StateDirectory and ConfigurationDirectory. This would replace ReadWritePaths and the tmpfile rules for rootDir and confDir as systemd will create those dirs automatically. I have set it like this:

systemd.services.crowdsec.serviceConfig = {
  StateDirectory = "crowdsec";
  StateDirectoryMode = "0750";
  ConfigurationDirectory = "crowdsec";
  ConfigurationDirectoryMode = "0750";
};

SebastianStork avatar Sep 28 '25 18:09 SebastianStork

People who use impermanence might appreciate DynamicUser being disabled (see https://github.com/nix-community/impermanence/issues/254). The things DynamicUser does behind the scenes, we already do explicitly (at least I think so? see https://unix.stackexchange.com/questions/635027/systemd-dynamicuser-vs-user)

SebastianStork avatar Sep 28 '25 18:09 SebastianStork

People who use impermanence might appreciate DynamicUser being disabled (see https://github.com/nix-community/impermanence/issues/254).

Hm... should I add an option where the user can enable/disable it or should I just remove the DynamicUser option? There was a discussion about the DynamicUser which you can read here https://github.com/NixOS/nixpkgs/pull/426875#discussion_r2218159097.

TornaxO7 avatar Sep 28 '25 18:09 TornaxO7

From what I have seen, the idiomatic way to make /var/lib/crowdsec and /etc/crowdsec accessible to the service, is to use the systemd service options StateDirectory and ConfigurationDirectory. This would replace ReadWritePaths and the tmpfile rules for rootDir and confDir as systemd will create those dirs automatically. I have set it like this:

After skimming through systemd.exec it looks like you're right. Nice one. Thanks!

TornaxO7 avatar Sep 28 '25 18:09 TornaxO7

People who use impermanence might appreciate DynamicUser being disabled (see nix-community/impermanence#254).

Hm... should I add an option where the user can enable/disable it or should I just remove the DynamicUser option? There was a discussion about the DynamicUser which you can read here #426875 (comment).

Hmm, I admit that I wasn't aware of this context. I have just been disabling anywhere it bothered me. I'd like some other people to way in on this. The impermanence issue is a bug and will hopefully be fixed sometime in the future, so that shouldn't be the only reason to forgo DynamicUser

SebastianStork avatar Sep 28 '25 19:09 SebastianStork

Anything I can do to help here? These options seem much more reasonable than the current ones.

I'd also like to mention I'm getting this error when trying to run it:

Error machine login for Haddock : ent: machine not found
127.0.0.1 - [Sun, 09 Nov 2025 15:55:06 PST] \"POST /v1/watchers/login HTTP/1.1 401 345.943µs \"crowdsec/v1.7.2-linux\" \"
attempt 1 out of 2
Error machine login for Haddock : ent: machine not found 
127.0.0.1 - [Sun, 09 Nov 2025 15:55:06 PST] \"POST /v1/watchers/login HTTP/1.1 401 123.611µs \"crowdsec/v1.7.2-linux\" \"
attempt 2 out of 2
Error machine login for Haddock : ent: machine not found
127.0.0.1 - [Sun, 09 Nov 2025 15:55:06 PST] \"POST /v1/watchers/login HTTP/1.1 401 100.691µs \"crowdsec/v1.7.2-linux\" \"
max attempts reached for status code 401
crowdsec init: while initializing LAPIClient: authenticate watcher (Haddock): API error: ent: machine not found

This is my current configuration: https://gist.github.com/poperigby/63ef3e5b6205b6a5e4de320bcb63838a

poperigby avatar Nov 09 '25 23:11 poperigby

Anything I can do to help here? These options seem much more reasonable than the current ones.

Aye, testing! What you already did:

I'd also like to mention I'm getting this error when trying to run it:

Thx!

I'll see when I'll get back to this

TornaxO7 avatar Nov 10 '25 10:11 TornaxO7

just tested this and ran into the same error as @poperigby. i looked in the journal and it seems like it needed the api.server.online_client.credentials_path set if anything needs to pull from crowdsec servers (anything on the hub, and possibly acquisitions) in addition the console setup fails and is a bit non intuitive. you need to go to the console and grab just the token out of the command it gives you and paste just that in the console.tokenFile. it then fails as it tries to write the console.yaml into /nix/store, which of course fails due to RO filesystem

here is my configuration repo with link pointed to my system flake and the line where the crowdsec specific bits start: https://gitlab.com/Amethyst_Nightshade/nixos-flakes/-/blob/main/nixosConfigurations/tailnerd-agh.nix#L304

AmethystNightshade avatar Nov 14 '25 17:11 AmethystNightshade

Alright, I'm back!

just tested this and ran into the same error as @poperigby. i looked in the journal and it seems like it needed the api.server.online_client.credentials_path set if anything needs to pull from crowdsec servers (anything on the hub, and possibly acquisitions) in addition the console setup fails and is a bit non intuitive. you need to go to the console and grab just the token out of the command it gives you and paste just that in the console.tokenFile. it then fails as it tries to write the console.yaml into /nix/store, which of course fails due to RO filesystem

Hm... this doesn't sound nice.

here is my configuration repo with link pointed to my system flake and the line where the crowdsec specific bits start: https://gitlab.com/Amethyst_Nightshade/nixos-flakes/-/blob/main/nixosConfigurations/tailnerd-agh.nix#L304

It looks like as if you've changed your repository. Could you please update the link?

TornaxO7 avatar Nov 28 '25 20:11 TornaxO7

sorry about the link, knew there was a reason i should have pinned the link at the commit. i had reorganized the repo breaking the link, but in doing so i had made the crowdsec config a module (among other stuff including a system change). the crowdsec module is used in the /nixosConfigurations/nerd0/system.nix which holds the system config itself. permalink at current state (as of comment): system git dir

AmethystNightshade avatar Dec 01 '25 15:12 AmethystNightshade

just remembered that it soft fails here for me as it explicitly checks for sudo, and i have sudo-rs as my sudo. i have not had any issues when i manually do that step with sudo-rs.

AmethystNightshade avatar Dec 05 '25 12:12 AmethystNightshade

just remembered that it soft fails here for me as it explicitly checks for sudo, and i have sudo-rs as my sudo. i have not had any issues when i manually do that step with sudo-rs.

Good catch! Fixed

TornaxO7 avatar Dec 10 '25 09:12 TornaxO7

you need to go to the console and grab just the token out of the command it gives you and paste just that in the console.tokenFile. it then fails as it tries to write the console.yaml into /nix/store, which of course fails due to RO filesystem

@AmethystNightshade hm... I'm unsure if I understand you correctly. So if you set the path in console.tokenFile you'll get an error because it tries to write to /nix/store but if you don't set console.tokenFile then everything works fine?

TornaxO7 avatar Dec 10 '25 10:12 TornaxO7

@AmethystNightshade hm... I'm unsure if I understand you correctly. So if you set the path in console.tokenFile you'll get an error because it tries to write to /nix/store but if you don't set console.tokenFile then everything works fine?

correct, when set, it tries to modify the console.yaml in /nix/store, doubt there is any way to force it not to short of modifying crowdsec, have not checked to see what got modified when i did a quick hacky patch here

AmethystNightshade avatar Dec 10 '25 11:12 AmethystNightshade

correct, when set, it tries to modify the console.yaml in /nix/store, doubt there is any way to force it not to short of modifying crowdsec, have not checked to see what got modified when i did a quick hacky patch here

Hm... I'm thinking about moving the console.yaml to /var/lib/crowdsec then.

TornaxO7 avatar Dec 10 '25 12:12 TornaxO7

Oh god, yeah, I see it now as well. Thank you for pointing that out! Should be fixed now

TornaxO7 avatar Dec 10 '25 14:12 TornaxO7

Alright, I'd say that the last known piece would be this huge chunk. Then I'd say that I'll rebase it and clean it up and then it's time for reviewing.

TornaxO7 avatar Dec 10 '25 18:12 TornaxO7

@NixOS/nix-formatting I'm getting the following error message if I do nix fmt:

error:
       … while evaluating an expression to select 'drvPath' on it
         at «internal»:1:552:
       … while evaluating strict
         at «internal»:1:552:
       (stack trace truncated; use '--show-trace' to show the full trace)

       error: hash mismatch in file downloaded from 'https://biomejs.dev/schemas/2.1.2/schema.json':
         specified: sha256-Qqd0XcWs54rigdyGXFysif32MBzbomWI0jkpSUeZFB8=
         got:       sha256-n4Y16J7g34e0VdQzRItu/P7n5oppkY4Vm4P1pQxOILU=

If I try to rebase with git rebase upstream/master then I'm getting a ton of merge conflicts. Any suggestions what I should do?

TornaxO7 avatar Dec 11 '25 14:12 TornaxO7

@TornaxO7 rebasing to master should fix the issue.

I was able to reproduce this issue on e7ab48b0bc834494033b36e671b095ae7e039060 (the last nixpkgs commit on this pr's branch)

HEAD is now at e7ab48b0bc83 python3Packages.baize: 0.22.2 -> 0.23.1 (#445851)
➜ nixpkgs (e7ab48b0bc83) ✔ nix fmt
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'treefmt'
         whose name attribute is located at /nix/store/lqh77v3g5x9ais0wxagnibrw3sjw2019-source/pkgs/stdenv/generic/make-derivation.nix:539:13

       … while evaluating attribute 'text' of derivation 'treefmt'
         at /nix/store/lqh77v3g5x9ais0wxagnibrw3sjw2019-source/pkgs/build-support/trivial-builders/default.nix:129:13:
          128|           inherit
          129|             text
             |             ^
          130|             executable

       … while evaluating the option `build.configFile':

       … while evaluating the option `settings.formatter.biome.options':

       … while evaluating definitions from `/nix/store/cahbgx59xna1qkacd5i63zkwg1jyvqkg-source/programs/biome.nix':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: hash mismatch in file downloaded from 'https://biomejs.dev/schemas/2.1.2/schema.json':
         specified: sha256:07qlk53lja9rsa46b8nv3hqgdzc9mif5r1nwh7i8mrxcqmfp99s2
         got:       sha256:1d909q6abxc3kcaqx4b9ibkfgzpwds5l8cylans8gpz0kvl3b1lz
➜ nixpkgs (e7ab48b0bc83) ✔

nix fmt is working fine on master

dyegoaurelio avatar Dec 11 '25 15:12 dyegoaurelio

Ok, just a simple git rebase upstream/master would require me to resolve probably multiple hundred files (for whatever reason). So I did a quick hack and it should be fixed now xD

TornaxO7 avatar Dec 11 '25 16:12 TornaxO7

Alright, so basically I'd like to get some feedback by trying out the new module regarding UX.

TornaxO7 avatar Dec 11 '25 16:12 TornaxO7

Oh god... I need to iterate through all crowdsec issues again and see if they are solved...

TornaxO7 avatar Dec 11 '25 16:12 TornaxO7

Hm... now that I think about it, I think it's better to do that incrementally. So let's say that this is just a refactor PR.

TornaxO7 avatar Dec 11 '25 17:12 TornaxO7

@06kellyjac could you please review it? :>

TornaxO7 avatar Dec 11 '25 17:12 TornaxO7

Hi, I don't have an overview of all the changes, but I would like to leave a comment: If Prometheus is disabled by default, the command cscli metrics will no longer work (source). This should be mentioned when setting it to false by default.

Peronia avatar Dec 12 '25 06:12 Peronia

Hi, I don't have an overview of all the changes, but I would like to leave a comment: If Prometheus is disabled by default, the command cscli metrics will no longer work (https://github.com/NixOS/nixpkgs/issues/469519#issuecomment-3637362136). This should be mentioned when setting it to false by default.

Oh, good catch hm... I think I'll set it back to true. Thank you for pointing that out!

TornaxO7 avatar Dec 12 '25 11:12 TornaxO7

While you're refactoring, there's a spelling error from the old version here: localAcquisisionFile = pkgs.writeText "local_acquisisions.yaml". This should be spelled "acquisitions". I'm also running into the issue there where it generates JSON instead of YAML.

OrionOth avatar Dec 13 '25 06:12 OrionOth