Allow specifying configurations to download in `crowdsec.yaml`

[poperigby]

What would you like to be added?

/kind feature

Allow specifying a list of configurations to be automatically downloaded in crowdsec.yaml.

Why is this needed?

This would be very useful for declarative setups (NixOS), and backing up your configuration. You wouldn't have to use a script to re-install the configurations you had before, but could simply load up CrowdSec with your configuration file.

[github-actions[bot]]

@poperigby: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions[bot]]

@poperigby: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind bug
• /kind packaging

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[poperigby]

/kind feature

[mmetc]

Hi,

we actually do have something for declarative installation of both hub items and acquisition files, but it works a bit different than what you propose. It requires calling "cscli" with a yaml file, after setting a feature flag.

# cat setup.yaml
setup:
- install:
    collections:
    - crowdsecurity/linux
  datasource:
    filenames:
    - /var/log/syslog
    - /var/log/kern.log
    - /var/log/messages
    labels:
      type: syslog
      source: file
- install:
    parsers:
    - crowdsecurity/whitelists
# CROWDSEC_FEATURE_CSCLI_SETUP=true ./test/local/bin/cscli setup install-hub ./setup.yaml
INFO Downloaded crowdsecurity/syslog-logs         
INFO Enabled parsers: crowdsecurity/syslog-logs   
INFO Downloaded crowdsecurity/geoip-enrich        
INFO Enabled parsers: crowdsecurity/geoip-enrich
[...]

The above command

• is quite reliable if you create the setup.yaml file yourself. Still a command to run before crowdsec but not a whole script. Try it with an invalid yaml to see in-context validation errors. There is a separate "setup validate" command that can be run before install-hub.

• is part of a more complex command that uses a set of rules to detect operating system and running services, we could use it in the default installation methods, but detecting the location of log files is not reliable and could conflict with users' existing configuration.

• is behind a feature flag which is unlikely to be removed soon but can change in the feature. We welcome any suggestion. See the tests in https://github.com/crowdsecurity/crowdsec/blob/master/test/bats/07_setup.bats and an example of the detection rules in https://github.com/crowdsecurity/crowdsec/blob/master/config/detect.yaml

• as for the other part - detecting running services -- it requires rules for each collection and it's quite easy with systemd, openrc (not implemented) or if the process name is known. Detecting running containers on the other hand, is easy to do wrong...

[poperigby]

Ah, that seems useful. It would be very helpful to have the daemon read that configuration file instead of having to use the CLI for it though.