[cscli/crowdsec] Test environment improvements

[LaurenceJJones]

What would you like to be added?

Currently if a user wants to download a test environment its pretty hard to get it configured correctly here are the issue:

• [ ] Cscli binary hubtest and explain evoke CrowdSec from path and this may not be the case for test environments
• [ ] Cscli binary hubtest evokes cscli from path and this may not be the case for test environments
• [ ] Cscli binary explain evokes CrowdSec with -no-api arguments meaning that it will FATA since it in a test env it most likely not running as a service unable to start crowdsec routines: authenticate watcher (test): Post \"http://127.0.0.1:8081/v1/watchers/login\" (#3385 )
• [x] Previous #3182 means patterns dir is not respected
• [ ] Invoking ./test_env.sh should replace configurations with full paths instead of relative directories

/kind enhancement

Why is this needed?

Easier to setup a dev / test environment

Workarounds

[!IMPORTANT]
You MUST run these commands within the tests directory inside the generated via test_env.sh unless stated otherwise.

Patterns dir issue

Run the following command to generate an /etc/crowdsec folder and proceed to symbolic link it to the test env

mkdir -p /etc/crowdsec/
ln -s "$PWD/config/patterns" /etc/crowdsec/patterns

Confirm the symbolic link is correct by running:

ls -la /etc/crowdsec/patterns/

Relative configuration options issue

You can sed find and replace ./ to the $PWD via:

sed -i "s|\./|$PWD/|g" dev.yaml

When running cat dev.yaml you should now see full paths for all options within the yaml

Hubtest cscli and crowdsec binary missing from path

Hubtest support providing crowdsec and cscli flags to the command if you followed our instructions on creating a developer environment if you have csdev alias setup you can run the following within the hub directory

csdev hubtest --cscli "$(dirname $PWD)/cscli" --crowdsec "$(dirname $PWD)/crowdsec" run --all

if you dont have csdev alias then run:

../cscli -c "$(dirname $PWD)/dev.yaml" hubtest --cscli "$(dirname $PWD)/cscli" --crowdsec "$(dirname $PWD)/crowdsec" run --all

[github-actions[bot]]

@LaurenceJJones: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions[bot]]

@LaurenceJJones: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind bug
• /kind packaging

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[Natsui31]

Example of bug repro from fresh debian 12

apt update \
  && apt install -y gettext curl wget git libsqlite3-0
cd $(mktemp -d)
wget https://github.com/crowdsecurity/crowdsec/releases/download/v1.6.2/crowdsec-release.tgz
tar xvzf crowdsec-release.tgz
cd crowdsec-v1.6.2
./test_env.sh
export PATH=$PATH:"$PWD/cmd/crowdsec-cli/":"$PWD/cmd/crowdsec/"
alias csdev="$PWD/cmd/crowdsec-cli/cscli -c $PWD/config/dev.yaml"
git clone https://github.com/crowdsecurity/hub
export PATH=$PATH:"$PWD/cmd/crowdsec/":"$PWD/cmd/crowdsec-cli/"
cd hub
csdev hubtest run --all

With this environment, we got the same error as described in [cscli] We added patterns_dir option to config.yaml but isnt used in hubtest On my test environment, config_paths.pattern_dir is not set.

root@05391e978e44:/tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/hub# csdev hubtest run --all
+ /tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/cmd/crowdsec-cli/cscli -c /tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/config/dev.yaml hubtest run --all
INFO Running test 'CVE-2017-9841'
ERRO running test 'CVE-2017-9841' failed: unable to copy 'patterns' from '/etc/crowdsec/patterns' to '/tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/hub/.tests/CVE-2017-9841/runtime/patterns': open /etc/crowdsec/patterns: no such file or directory
INFO Running test 'CVE-2019-18935'
ERRO running test 'CVE-2019-18935' failed: unable to copy 'patterns' from '/etc/crowdsec/patterns' to '/tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/hub/.tests/CVE-2019-18935/runtime/patterns': open /etc/crowdsec/patterns: no such file or directory
INFO Running test 'CVE-2021-4034'
ERRO running test 'CVE-2021-4034' failed: unable to copy 'patterns' from '/etc/crowdsec/patterns' to '/tmp/tmp.MqISOY3GSv/crowdsec-v1.6.2/hub/.tests/CVE-2021-4034/runtime/patterns': open /etc/crowdsec/patterns: no such file or directory
...

I assume that behavior could be disruptive when testing:

• Upgrade compatibility to newer major version
• Older major version installed with newer major version environment test

Note: I expect the test environment to be isolated from any pre-existing installations, ensuring that the results are not influenced by previously installed versions.

[sdwilsh]

Invoking ./test_env.sh should replace configurations with full paths instead of relative directories

Having this would have made it much easier to create a devcontainer for parsers/scenarios in https://github.com/crowdsecurity/hub/pull/1256. Once I realized it was configurable (because I found this issue), I was able to update it to make a devcontainer possible, but having full paths in the config would have greatly cut down the time I spent chasing this down.

[mmetc]

full paths in config: done 1.7.1

@LaurenceJJones not sure what you need for 1) and 2)

"cscli hubtest" and "explain" have --crowdsec and --cscli parameters

I usually prefer the bats environment which is more isolated and comes with its own env script to be sourced, it's more complex to understand but some ideas can be borrowed.

Another solution is like python's activate or aws-shell, open a subshell with the modified PATH and other variables if required.