crowdsec icon indicating copy to clipboard operation
crowdsec copied to clipboard
verified publisher icon crowdsecurity

Running hub test under v1.6.9 is broken due to missing path

Open nvtkaszpir opened this issue 6 months ago • 4 comments

What happened?

Following https://doc.crowdsec.net/docs/contributing/contributing_test_env/

What did you expect to happen?

Running ../cscli -c ../dev.yaml hubtest run --all would execute testst, but it fails to copy non-exisitng path.

How can we reproduce it (as minimally and precisely as possible)?

15:57:14 kaszpir@lynx ~/src $ VER=1.6.9
0
15:57:32 kaszpir@lynx ~/src $ wget https://github.com/crowdsecurity/crowdsec/releases/download/v$VER/crowdsec-release.tgz
tar xvzf crowdsec-release.tgz
cd crowdsec-v$VER
--2025-06-22 15:57:39--  https://github.com/crowdsecurity/crowdsec/releases/download/v1.6.9/crowdsec-release.tgz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/264154402/e5df48d7-eaf2-48ec-ab09-d4626f05680f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250622%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250622T135739Z&X-Amz-Expires=1800&X-Amz-Signature=d282e199d64e942679b0d8177f03094ba10c97e1b5b144c84a951a3c38dd3f36&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dcrowdsec-release.tgz&response-content-type=application%2Foctet-stream [following]
--2025-06-22 15:57:39--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/264154402/e5df48d7-eaf2-48ec-ab09-d4626f05680f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250622%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250622T135739Z&X-Amz-Expires=1800&X-Amz-Signature=d282e199d64e942679b0d8177f03094ba10c97e1b5b144c84a951a3c38dd3f36&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dcrowdsec-release.tgz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 121673974 (116M) [application/octet-stream]
Saving to: ‘crowdsec-release.tgz.1’

crowdsec-release.tgz.1                                    100%[====================================================================================================================================>] 116,04M  16,3MB/s    in 8,1s    

2025-06-22 15:57:47 (14,2 MB/s) - ‘crowdsec-release.tgz.1’ saved [121673974/121673974]

crowdsec-v1.6.9/
crowdsec-v1.6.9/config/
crowdsec-v1.6.9/config/config_win_no_lapi.yaml
crowdsec-v1.6.9/config/crowdsec.service
crowdsec-v1.6.9/config/console.yaml
crowdsec-v1.6.9/config/acquis_win.yaml
crowdsec-v1.6.9/config/config_win.yaml
crowdsec-v1.6.9/config/dev.yaml
crowdsec-v1.6.9/config/simulation.yaml
crowdsec-v1.6.9/config/local_api_credentials.yaml
crowdsec-v1.6.9/config/context.yaml
crowdsec-v1.6.9/config/detect.yaml
crowdsec-v1.6.9/config/acquis.yaml
crowdsec-v1.6.9/config/online_api_credentials.yaml
crowdsec-v1.6.9/config/user.yaml
crowdsec-v1.6.9/config/profiles.yaml
crowdsec-v1.6.9/config/config.yaml
crowdsec-v1.6.9/config/patterns/
crowdsec-v1.6.9/config/patterns/ssh
crowdsec-v1.6.9/config/patterns/modsecurity
crowdsec-v1.6.9/config/patterns/bro
crowdsec-v1.6.9/config/patterns/linux-syslog
crowdsec-v1.6.9/config/patterns/paths
crowdsec-v1.6.9/config/patterns/mcollective
crowdsec-v1.6.9/config/patterns/exim
crowdsec-v1.6.9/config/patterns/rails
crowdsec-v1.6.9/config/patterns/smb
crowdsec-v1.6.9/config/patterns/postgresql
crowdsec-v1.6.9/config/patterns/bacula
crowdsec-v1.6.9/config/patterns/nagios
crowdsec-v1.6.9/config/patterns/ruby
crowdsec-v1.6.9/config/patterns/nginx
crowdsec-v1.6.9/config/patterns/aws
crowdsec-v1.6.9/config/patterns/firewalls
crowdsec-v1.6.9/config/patterns/cowrie_honeypot
crowdsec-v1.6.9/config/patterns/mongodb
crowdsec-v1.6.9/config/patterns/haproxy
crowdsec-v1.6.9/config/patterns/junos
crowdsec-v1.6.9/config/patterns/tcpdump
crowdsec-v1.6.9/config/patterns/mysql
crowdsec-v1.6.9/config/patterns/java
crowdsec-v1.6.9/config/patterns/redis
crowdsec-v1.6.9/config/crowdsec.cron.daily
crowdsec-v1.6.9/cmd/
crowdsec-v1.6.9/cmd/notification-email/
crowdsec-v1.6.9/cmd/notification-email/notification-email
crowdsec-v1.6.9/cmd/notification-email/email.yaml
crowdsec-v1.6.9/cmd/notification-http/
crowdsec-v1.6.9/cmd/notification-http/http.yaml
crowdsec-v1.6.9/cmd/notification-http/notification-http
crowdsec-v1.6.9/cmd/notification-splunk/
crowdsec-v1.6.9/cmd/notification-splunk/notification-splunk
crowdsec-v1.6.9/cmd/notification-splunk/splunk.yaml
crowdsec-v1.6.9/cmd/crowdsec-cli/
crowdsec-v1.6.9/cmd/crowdsec-cli/cscli
crowdsec-v1.6.9/cmd/notification-dummy/
crowdsec-v1.6.9/cmd/notification-dummy/notification-dummy
crowdsec-v1.6.9/cmd/notification-dummy/dummy.yaml
crowdsec-v1.6.9/cmd/notification-slack/
crowdsec-v1.6.9/cmd/notification-slack/notification-slack
crowdsec-v1.6.9/cmd/notification-slack/slack.yaml
crowdsec-v1.6.9/cmd/notification-sentinel/
crowdsec-v1.6.9/cmd/notification-sentinel/sentinel.yaml
crowdsec-v1.6.9/cmd/notification-sentinel/notification-sentinel
crowdsec-v1.6.9/cmd/notification-file/
crowdsec-v1.6.9/cmd/notification-file/file.yaml
crowdsec-v1.6.9/cmd/notification-file/notification-file
crowdsec-v1.6.9/cmd/crowdsec/
crowdsec-v1.6.9/cmd/crowdsec/crowdsec
crowdsec-v1.6.9/test_env.ps1
crowdsec-v1.6.9/wizard.sh
crowdsec-v1.6.9/test_env.sh
0
15:57:49 kaszpir@lynx ~/src/crowdsec-v1.6.9 $ ./test_env.sh
[22/06/25:15:57:56][INFO] Creating test tree in /home/kaszpir/src/crowdsec-v1.6.9/tests
[22/06/25:15:57:56][INFO] Tree created
[22/06/25:15:57:56][INFO] Copying needed files for tests environment
[22/06/25:15:57:56][INFO] Files copied
[22/06/25:15:57:56][INFO] Setting up configurations
WARNING can't load CAPI credentials from './config/online_api_credentials.yaml' (missing login field) 
Machine 'test' successfully added to the local API.
API credentials written to '/home/kaszpir/src/crowdsec-v1.6.9/tests/config/local_api_credentials.yaml'.
Downloading /home/kaszpir/src/crowdsec-v1.6.9/tests/config/hub/.index.json
Action plan:
📥 download
 collections: crowdsecurity/linux (0.2), crowdsecurity/sshd (0.7)
 contexts: crowdsecurity/bf_base (0.1)
 scenarios: crowdsecurity/ssh-bf (0.3), crowdsecurity/ssh-cve-2024-6387 (0.2), crowdsecurity/ssh-generic-test (0.2), crowdsecurity/ssh-refused-conn (0.1), crowdsecurity/ssh-slow-bf (0.4)
 parsers: crowdsecurity/dateparse-enrich (0.2), crowdsecurity/geoip-enrich (0.5), crowdsecurity/sshd-logs (3.0), crowdsecurity/syslog-logs (0.8)
✅ enable
 collections: crowdsecurity/linux, crowdsecurity/sshd
 contexts: crowdsecurity/bf_base
 scenarios: crowdsecurity/ssh-bf, crowdsecurity/ssh-cve-2024-6387, crowdsecurity/ssh-generic-test, crowdsecurity/ssh-refused-conn, crowdsecurity/ssh-slow-bf
 parsers: crowdsecurity/dateparse-enrich, crowdsecurity/geoip-enrich, crowdsecurity/sshd-logs, crowdsecurity/syslog-logs

downloading parsers:crowdsecurity/syslog-logs
downloading parsers:crowdsecurity/geoip-enrich
downloading https://hub-data.crowdsec.net/mmdb_update/GeoLite2-City.mmdb
downloading https://hub-data.crowdsec.net/mmdb_update/GeoLite2-ASN.mmdb
downloading parsers:crowdsecurity/dateparse-enrich
downloading parsers:crowdsecurity/sshd-logs
downloading scenarios:crowdsecurity/ssh-bf
downloading scenarios:crowdsecurity/ssh-slow-bf
downloading scenarios:crowdsecurity/ssh-cve-2024-6387
downloading scenarios:crowdsecurity/ssh-refused-conn
downloading scenarios:crowdsecurity/ssh-generic-test
downloading contexts:crowdsecurity/bf_base
downloading collections:crowdsecurity/sshd
downloading collections:crowdsecurity/linux
enabling parsers:crowdsecurity/syslog-logs
enabling parsers:crowdsecurity/geoip-enrich
enabling parsers:crowdsecurity/dateparse-enrich
enabling parsers:crowdsecurity/sshd-logs
enabling scenarios:crowdsecurity/ssh-bf
enabling scenarios:crowdsecurity/ssh-slow-bf
enabling scenarios:crowdsecurity/ssh-cve-2024-6387
enabling scenarios:crowdsecurity/ssh-refused-conn
enabling scenarios:crowdsecurity/ssh-generic-test
enabling contexts:crowdsecurity/bf_base
enabling collections:crowdsecurity/sshd
enabling collections:crowdsecurity/linux

Run 'sudo systemctl reload crowdsec' for the new configuration to be effective.
[22/06/25:15:57:58][INFO] Environment is ready in /home/kaszpir/src/crowdsec-v1.6.9/tests
0
15:57:59 kaszpir@lynx ~/src/crowdsec-v1.6.9 $ cd tests
0
15:58:10 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ ./crowdsec -c dev.yaml
WARN[2025-06-22T15:58:14+02:00] can't load CAPI credentials from './config/online_api_credentials.yaml' (missing login field) 
INFO[2025-06-22T15:58:14+02:00] push and pull to Central API disabled        
INFO[2025-06-22T15:58:14+02:00] Enabled feature flags: none                  
INFO[2025-06-22T15:58:14+02:00] Crowdsec v1.6.9-40b8cfe6                     
INFO[2025-06-22T15:58:14+02:00] Loading prometheus collectors                
WARN[2025-06-22T15:58:14+02:00] Communication with CrowdSec Central API disabled from configuration file 
INFO[2025-06-22T15:58:14+02:00] push and pull to Central API disabled        
INFO[2025-06-22T15:58:14+02:00] CrowdSec Local API listening on 127.0.0.1:8081 
INFO[2025-06-22T15:58:14+02:00] Loading grok library /home/kaszpir/src/crowdsec-v1.6.9/tests/config/patterns 
INFO[2025-06-22T15:58:15+02:00] Loading enrich plugins                       
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'GeoIpCity' 
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'GeoIpASN'  
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'IpToRange' 
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'reverse_dns' 
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'ParseDate' 
INFO[2025-06-22T15:58:15+02:00] Successfully registered enricher 'UnmarshalJSON' 
INFO[2025-06-22T15:58:15+02:00] Loading parsers from 4 files                 
INFO[2025-06-22T15:58:15+02:00] Loaded 2 parser nodes                         file=/home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s00-raw/syslog-logs.yaml stage=s00-raw
INFO[2025-06-22T15:58:15+02:00] Loaded 1 parser nodes                         file=/home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s01-parse/sshd-logs.yaml stage=s01-parse
INFO[2025-06-22T15:58:15+02:00] Loaded 1 parser nodes                         file=/home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s02-enrich/dateparse-enrich.yaml stage=s02-enrich
INFO[2025-06-22T15:58:15+02:00] Loaded 1 parser nodes                         file=/home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s02-enrich/geoip-enrich.yaml stage=s02-enrich
INFO[2025-06-22T15:58:15+02:00] Loaded 5 nodes from 3 stages                 
INFO[2025-06-22T15:58:15+02:00] No postoverflow parsers to load              
INFO[2025-06-22T15:58:15+02:00] Loading 5 scenario files                     
INFO[2025-06-22T15:58:15+02:00] Adding trigger bucket                         cfg=small-fog name=crowdsecurity/ssh-generic-test
INFO[2025-06-22T15:58:15+02:00] Adding leaky bucket                           cfg=autumn-resonance name=crowdsecurity/ssh-bf
INFO[2025-06-22T15:58:15+02:00] Adding leaky bucket                           cfg=withered-morning name=crowdsecurity/ssh-bf_user-enum
INFO[2025-06-22T15:58:15+02:00] Adding leaky bucket                           cfg=little-glade name=crowdsecurity/ssh-cve-2024-6387
INFO[2025-06-22T15:58:15+02:00] Adding trigger bucket                         cfg=purple-fog name=crowdsecurity/ssh-refused-conn
INFO[2025-06-22T15:58:15+02:00] Adding leaky bucket                           cfg=icy-rain name=crowdsecurity/ssh-slow-bf
INFO[2025-06-22T15:58:15+02:00] Adding leaky bucket                           cfg=crimson-glade name=crowdsecurity/ssh-slow-bf_user-enum
INFO[2025-06-22T15:58:15+02:00] Loaded 7 scenarios                           
INFO[2025-06-22T15:58:15+02:00] 127.0.0.1 - [Sun, 22 Jun 2025 15:58:15 CEST] "POST /v1/watchers/login HTTP/1.1 200 60.629175ms "crowdsec/v1.6.9-40b8cfe6-linux" " 
INFO[2025-06-22T15:58:15+02:00] loading acquisition file : /home/kaszpir/src/crowdsec-v1.6.9/tests/config/acquis.yaml 
WARN[2025-06-22T15:58:15+02:00] No matching files for pattern /var/log/nginx/*.log  type=file
WARN[2025-06-22T15:58:15+02:00] No matching files for pattern ./tests/nginx/nginx.log  type=file
INFO[2025-06-22T15:58:15+02:00] Adding file /var/log/auth.log to datasources  type=file
INFO[2025-06-22T15:58:15+02:00] Adding file /var/log/syslog to datasources    type=file
WARN[2025-06-22T15:58:15+02:00] No matching files for pattern /var/log/apache2/*.log  type=file
WARN[2025-06-22T15:58:15+02:00] prometheus: listen tcp 127.0.0.1:6060: bind: address already in use 
INFO[2025-06-22T15:58:15+02:00] Starting processing data                     
INFO[2025-06-22T15:58:15+02:00] 127.0.0.1 - [Sun, 22 Jun 2025 15:58:15 CEST] "POST /v1/usage-metrics HTTP/1.1 201 519.485µs "crowdsec/v1.6.9-40b8cfe6-linux" " 
^CWARN[2025-06-22T15:58:22+02:00] SIGTERM received, shutting down              
INFO[2025-06-22T15:58:22+02:00] Crowdsec engine shutting down                
INFO[2025-06-22T15:58:22+02:00] File datasource stopping                      tail=/var/log/auth.log type=file
INFO[2025-06-22T15:58:22+02:00] File datasource stopping                      tail=/var/log/syslog type=file
INFO[2025-06-22T15:58:22+02:00] Killing parser routines                      
INFO[2025-06-22T15:58:24+02:00] Bucket routine exiting                       
INFO[2025-06-22T15:58:25+02:00] serve: shutting down api server              
INFO[2025-06-22T15:58:25+02:00] pluginTomb dying                             
INFO[2025-06-22T15:58:25+02:00] killing all plugins                          
INFO[2025-06-22T15:58:25+02:00] Shutting down API server                     
WARN[2025-06-22T15:58:25+02:00] Crowdsec service shutting down               
0
15:58:25 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ ./cscli -c dev.yaml hub list
Loaded: 142 parsers, 10 postoverflows, 764 scenarios, 8 contexts, 4 appsec-configs, 116 appsec-rules, 139 collections
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 PARSERS                                                                                                                                       
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 Name                            📦 Status    Version  Local Path                                                                              
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 crowdsecurity/dateparse-enrich  ✔️  enabled  0.2      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s02-enrich/dateparse-enrich.yaml 
 crowdsecurity/geoip-enrich      ✔️  enabled  0.5      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s02-enrich/geoip-enrich.yaml     
 crowdsecurity/sshd-logs         ✔️  enabled  3.0      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s01-parse/sshd-logs.yaml         
 crowdsecurity/syslog-logs       ✔️  enabled  0.8      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/parsers/s00-raw/syslog-logs.yaml         
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 SCENARIOS                                                                                                                              
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 Name                             📦 Status    Version  Local Path                                                                      
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 crowdsecurity/ssh-bf             ✔️  enabled  0.3      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/scenarios/ssh-bf.yaml            
 crowdsecurity/ssh-cve-2024-6387  ✔️  enabled  0.2      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/scenarios/ssh-cve-2024-6387.yaml 
 crowdsecurity/ssh-generic-test   ✔️  enabled  0.2      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/scenarios/ssh-generic-test.yaml  
 crowdsecurity/ssh-refused-conn   ✔️  enabled  0.1      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/scenarios/ssh-refused-conn.yaml  
 crowdsecurity/ssh-slow-bf        ✔️  enabled  0.4      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/scenarios/ssh-slow-bf.yaml       
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 CONTEXTS                                                                                                          
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 Name                   📦 Status    Version  Local Path                                                           
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 crowdsecurity/bf_base  ✔️  enabled  0.1      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/contexts/bf_base.yaml 
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 COLLECTIONS                                                                                                      
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 Name                 📦 Status    Version  Local Path                                                            
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 crowdsecurity/linux  ✔️  enabled  0.2      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/collections/linux.yaml 
 crowdsecurity/sshd   ✔️  enabled  0.7      /home/kaszpir/src/crowdsec-v1.6.9/tests/config/collections/sshd.yaml  
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────
0
15:58:28 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ git clone https://github.com/crowdsecurity/hub
Cloning into 'hub'...
remote: Enumerating objects: 27761, done.
remote: Counting objects: 100% (166/166), done.
remote: Compressing objects: 100% (75/75), done.
remote: Total 27761 (delta 128), reused 102 (delta 90), pack-reused 27595 (from 3)
Receiving objects: 100% (27761/27761), 197.28 MiB | 41.64 MiB/s, done.
Resolving deltas: 100% (17704/17704), done.
0
15:58:45 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ cd hub
0
15:58:47 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests/hub (master) $ 
Running all tests (max_jobs: 12)
Running test 'CVE-2017-9841'
Running test 'CVE-2019-18935'
Running test 'CVE-2021-4034'
Running test 'CVE-2022-26134'
Running test 'CVE-2022-35914'
Running test 'CVE-2022-40684'
Running test 'CVE-2022-41697'
Running test 'CVE-2022-42889'
Running test 'CVE-2022-44877'
Running test 'CVE-2022-46169'
Running test 'CVE-2023-22515'
Running test 'CVE-2023-22518'
Running test 'CVE-2023-4911'
Running test 'CVE-2024-0012'
Running test 'CVE-2024-38475'
Running test 'CVE-2024-9474'
Running test 'CVE-2025-0108'
Running test 'adguardhome-bf'
Running test 'adguardhome-logs'
Running test 'amavis-blocked'
Running test 'amavis-logs'
Running test 'apache-cve-2021-41773'
Running test 'apache-guacamole-logs'
Running test 'apache-guacamole_bf'
Running test 'apache-guacamole_user_enum'
Running test 'apache-http-probing'
Running test 'apache2-http-sensitive-files'
Running test 'apache2-logs'
Running test 'apache2-malformed'
Running test 'apache_log4j2_cve-2021-44228'
Running test 'apereo-cas-audit-logs'
Running test 'apereo-cas-bf'
Running test 'apereo-cas-slow-bf'
Running test 'apiscp-bf'
Running test 'asterisk-bf'
Running test 'asterisk-logs'
Running test 'asterisk-syslogs'
Running test 'asterisk-user-enum'
Running test 'audiobookshelf-bf'
Running test 'audiobookshelf-logs'
Running test 'auditd-base64-exec'
Running test 'auditd-logs'
Running test 'auditd-postexploit-exec-from-net'
Running test 'auditd-postexploit-rm'
Running test 'auditd-suid-crash'
Running test 'authelia-bf'
Running test 'authelia-logs'
Running test 'authentik-bf'
Running test 'authentik-logs'
Running test 'aws-alb-logs'
Running test 'aws-bf'
Running test 'aws-cloudfront-logs'
Running test 'aws-cloudtrail'
Running test 'aws-cloudtrail-postexploit'
Running test 'aws-nwo-login'
Running test 'baikal-bf'
Running test 'baikal-logs'
Running test 'baserow-logs'
Running test 'bind9-logs'
Running test 'bind9-refused'
Running test 'bind9-syslog'
Running test 'bitwarden-bf'
Running test 'bitwarden-logs'
Running test 'bookstack-bf'
Running test 'bookstack-logs'
Running test 'caddy-basic-auth-bf'
Running test 'caddy-coraza'
Running test 'caddy-crs-anomaly-score'
Running test 'caddy-logs'
Running test 'charon-ipsec-bf'
Running test 'charon-ipsec-logs'
Running test 'charon-ipsec-slow-bf'
Running test 'configserver-lfd-logs'
Running test 'couchdb-logs'
Running test 'cowrie-logs'
Running test 'cowrie_telnet-bf'
Running test 'cpanel-bf'
Running test 'cpanel-logs'
Running test 'cpanel_bf_attempt'
Running test 'cri-logs'
Running test 'cve-2021-42013'
Running test 'cve-2023-23397'
Running test 'cve-2023-49103'
Running test 'cve_2022_37042'
Running test 'docker-logs'
Running test 'dockge-bf'
Running test 'dockge-logs'
Running test 'dovecot-logs'
Running test 'dovecot-spam'
Running test 'dropbear-logs'
Running test 'dropbear-ssh-bf'
Running test 'emby-bf'
Running test 'emby-logs'
Running test 'endlessh-logs'
Running test 'endlessh-syslogs'
Running test 'exchange-imap-bf'
Running test 'exchange-imap-logs'
Running test 'exchange-pop-bf'
Running test 'exchange-pop-logs'
Running test 'exchange-smtp-bf'
Running test 'exchange-smtp-logs'
Running test 'exim-bf'
Running test 'exim-logs'
Running test 'exim-spam'
Running test 'exim-syslog-logs'
Running test 'f5-big-ip-cve-2020-5902'
Running test 'fastly-logs'
Running test 'filebrowser-logs'
Running test 'fortinet-cve-2018-13379'
Running test 'fortinet-vpn-bf'
Running test 'freeswitch'
Running test 'freeswitch-acl-reject'
Running test 'freeswitch-bf'
Running test 'freeswitch-user-enumeration'
Running test 'geoip-enrich'
Running test 'gitea-bf'
Running test 'gitea-logs'
Running test 'gitlab-bf'
Running test 'gitlab-logs'
Running test 'gotify-bf'
Running test 'gotify-logs'
Running test 'grafana-bf'
Running test 'grafana-logs'
Running test 'grafana_cve-2021-43798'
Running test 'haproxy-logs'
Running test 'haproxy-nextcloud-whitelist'
Running test 'harbor-bf'
Running test 'harbor-logs'
Running test 'hestiacp-bf'
Running test 'hestiacp-logs'
Running test 'home-assistant'
Running test 'http-admin-interface-probing'
Running test 'http-bf-wordpress-bf'
Running test 'http-cve-probing'
Running test 'http-dos-bypass-cache'
Running test 'http-dos-invalid-http-versions'
Running test 'http-dos-random-uri'
Running test 'http-dos-switching-ua'
Running test 'http-generic-test'
Running test 'http-magento-bf'
Running test 'http-wordpress-scan'
Running test 'http-wordpress_user-enum'
Running test 'http-wordpress_wpconfig'
Running test 'iis-xml-logs'
Running test 'immich-bf'
Running test 'immich-logs'
Running test 'iptables-logs'
Running test 'iptables-scan-multi-port'
Running test 'ipv6-postoverflow'
Running test 'jellyfin-bf'
Running test 'jellyfin-logs'
Running test 'jellyfin-syslog-bf'
Running test 'jellyfin-syslog-logs'
Running test 'jellyfin-whitelist'
Running test 'jellyseerr-bf'
Running test 'jellyseerr-logs'
Running test 'jira_cve-2021-26086'
Running test 'joplin-server-bf'
Running test 'joplin-server-logs'
Running test 'k8s-audit-pod-exec-file'
Running test 'k8s-audit-priv-pod-file'
Running test 'kasm'
Running test 'kasm-bruteforce'
Running test 'keycloak-bf'
Running test 'keycloak-logs'
Running test 'keycloak-slow-bf'
Running test 'laurel-base64-exec'
Running test 'laurel-logs'
Running test 'laurel-suid-crash'
Running test 'lemonldap-ng-bf'
Running test 'lemonldap-ng-logs'
Running test 'litellm-logs'
Running test 'litespeed-admin-bf'
Running test 'litespeed-http-sensitive-files'
Running test 'litespeed-logs'
Running test 'magento-ccs'
Running test 'magento-ccs-by-as'
Running test 'magento-ccs-by-country'
Running test 'magento-extension-logs'
Running test 'mailu-admin-bf'
Running test 'mailu-admin-logs'
Running test 'mariadb-bf'
Running test 'mariadb-logs'
Running test 'meshcentral-bf'
Running test 'meshcentral-logs'
Running test 'mikrotik-bf'
Running test 'mikrotik-logs'
Running test 'mikrotik-scan-multi_ports'
Running test 'miniflux-bf'
Running test 'miniflux-logs'
Running test 'modsecurity'
Running test 'modsecurity-logs'
Running test 'modsecurity-logs-nginx'
Running test 'modsecurity-nginx'
Running test 'mongodb-bf'
Running test 'mongodb-logs'
Running test 'mssql-text-logs'
Running test 'mysql-bf'
Running test 'mysql-logs'
Running test 'navidrome-bf'
Running test 'navidrome-logs'
Running test 'netgear_rce'
Running test 'nextcloud-bf'
Running test 'nextcloud-logs'
Running test 'nextcloud-whitelist'
Running test 'nginx-bad-user-agent'
Running test 'nginx-cve-2021-41773'
Running test 'nginx-http-backdoor'
Running test 'nginx-http-generic-bf'
Running test 'nginx-http-malformed'
Running test 'nginx-http-open-proxy'
Running test 'nginx-http-path-traversal'
Running test 'nginx-http-sensitive-files'
Running test 'nginx-http-sqli-probing'
Running test 'nginx-http-w00twoot'
Running test 'nginx-http-xss-probing'
Running test 'nginx-mail-bf'
Running test 'nginx-mail-logs'
Running test 'nginx-proxy-manager-logs'
Running test 'nginx-proxy-manager-malformed'
Running test 'nginx_http-logs'
Running test 'nginx_req_limit_exceeded'
Running test 'npmplus-logs'
Running test 'odoo-bf_user-enum'
Running test 'odoo-logs'
Running test 'ombi-bf'
Running test 'ombi-logs'
Running test 'openappsec'
Running test 'opensearch-dashboard-bf'
Running test 'opensearch-dashboard-logs'
Running test 'openvpn'
Running test 'openvpn-bf'
Running test 'opnsense-gui-auth'
Running test 'opnsense-sshd'
Running test 'overseerr-bf'
Running test 'overseerr-logs'
Running test 'palo-alto-threat'
Running test 'pam-logs'
Running test 'paperless-ngx-bf'
Running test 'paperless-ngx-logs'
Running test 'pf-logs'
Running test 'pf-scan-multi-port'
Running test 'pfsense-gui-auth'
Running test 'pgsql-logs'
Running test 'pgsql-user-enum'
Running test 'postfix-helo'
Running test 'postfix-logs'
Running test 'postfix-non-smtp'
Running test 'postfix-relay'
Running test 'postfix-spam'
Running test 'postscreen-logs'
Running test 'proftpd-bf'
Running test 'proftpd-logs'
Running test 'proftpd-user-enum'
Running test 'prowlarr-bf'
Running test 'prowlarr-logs'
Running test 'prowlarr-nonsyslog-logs'
Running test 'proxmox-bf'
Running test 'proxmox-iptables-logs'
Running test 'proxmox-logs'
Running test 'pterodactyl-wings'
Running test 'pterodactyl-wings-bf'
Running test 'pulse-secure-sslvpn-cve-2019-11510'
Running test 'pureftpd-bf'
Running test 'pureftpd-logs'
Running test 'radarr-bf'
Running test 'radarr-logs'
Running test 'radarr-nonsyslog-logs'
Running test 'redmine-bf'
Running test 'redmine-logs'
Running test 'rocketchat-whitelist'
Running test 'sabnzbd-bf'
Running test 'sabnzbd-logs'
Running test 'sap-probing'
Running test 'segfault-logs'
Running test 'smb-bf'
Running test 'smb-logs'
Running test 'sonarr-bf'
Running test 'sonarr-logs'
Running test 'sonarr-nonsyslog-logs'
Running test 'spring4shell_cve-2022-22965'
Running test 'ssh-bf'
Running test 'ssh-generic-test'
Running test 'ssh-slow-bf'
Running test 'ssh-timeout'
Running test 'sshd-bad-keyexchange-bf'
Running test 'sshd-impossible-travel'
Running test 'sshd-impossible-travel-user'
Running test 'sshd-invalid-bf'
Running test 'sshd-logs'
Running test 'sshd-logs-fp'
Running test 'sshd-refused-conn'
Running test 'sshd-success-logs'
Running test 'sshd_banner_exchange'
Running test 'sshesame'
Running test 'stirling-pdf-bf'
Running test 'stirling-pdf-logs'
Running test 'supabase-docker-pgsql-logs'
Running test 'suricata-eve-detect'
Running test 'suricata-logs-evelog'
Running test 'suricata-logs-fastlog'
Running test 'synology-dsm-bf'
Running test 'synology-dsm-logs'
Running test 'syslog-logs'
Running test 'tcpdump-logs'
Running test 'tcpudp-flood-traefik'
Running test 'teamspeak-bf'
Running test 'teleport-bf'
Running test 'teleport-impossible-travel'
Running test 'teleport-logs'
Running test 'thehive-bf'
Running test 'thehive-logs'
Running test 'thinkphp-cve-2018-20062'
Running test 'traefik_base-http-scenario'
Running test 'traefik_clf'
Running test 'traefik_json'
Running test 'unifi-logs'
Running test 'uptime-kuma-bf'
Running test 'uptime-kuma-logs'
Running test 'vaultwarden-bf'
Running test 'vaultwarden-logs'
Running test 'vmware-cve-2022-22954'
Running test 'vmware-vcenter-vmsa-2021-0027'
Running test 'vsftpd-bf'
Running test 'vsftpd-logs'
Running test 'webmin-bf'
Running test 'webmin-logs'
Running test 'whitelists'
Running test 'windows-bf'
Running test 'windows-logs'
Running test 'wireguard-auth'
Running test 'wireguard-logs'
Running test 'zimbra-bf'
Running test 'zimbra-logs'
Running test 'zoneminder-bf'
Running test 'zoneminder-logs'
Running test 'zoneminder_cve-39285'
Running test 'zoneminder_cve-39290'
Running test 'zoneminder_cve-39291'
Running test 'zoraxy-http-bad-user-agent'
Running test 'zoraxy-http-logs'
Error: unable to copy 'patterns' from '/home/kaszpir/src/crowdsec-v1.6.9/tests/hub/config/patterns' to '/home/kaszpir/src/crowdsec-v1.6.9/tests/hub/.tests/CVE-2022-40684/runtime/patterns': stat .: no such file or directory
0
15:58:57 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests/hub (master) $

Anything else we need to know?

16:06:39 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ ./cscli  version
version: v1.6.9-40b8cfe6
Codename: alphaga
BuildDate: 2025-06-17_11:56:26
GoVersion: 1.24.3
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.9-40b8cfe6-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog
0
16:06:41 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ cscli version
version: v1.6.9-debian-pragmatic-amd64-40b8cfe6
Codename: alphaga
BuildDate: 2025-06-17_14:01:07
GoVersion: 1.24.3
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.9-debian-pragmatic-amd64-40b8cfe6-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

Crowdsec version

$ cscli version
version: v1.6.9-debian-pragmatic-amd64-40b8cfe6
Codename: alphaga
BuildDate: 2025-06-17_14:01:07
GoVersion: 1.24.3
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.9-debian-pragmatic-amd64-40b8cfe6-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

OS version

# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

$ uname -a
Linux lynx 6.8.0-59-generic #61~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 17:03:15 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here
# not tested on windows

Enabled collections and parsers

$ cscli hub list -o raw
16:02:37 kaszpir@lynx ~/src/crowdsec-v1.6.9/tests $ cscli -c dev.yaml hub list -o raw
Loaded: 142 parsers, 10 postoverflows, 764 scenarios, 8 contexts, 4 appsec-configs, 116 appsec-rules, 139 collections
name,status,version,description,type
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/sshd-logs,enabled,3.0,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios
crowdsecurity/ssh-generic-test,enabled,0.2,Crowdsec Generic Test Scenario: SSH brute force trigger,scenarios
crowdsecurity/ssh-refused-conn,enabled,0.1,Detect sshd refused connections,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios
crowdsecurity/bf_base,enabled,0.1,,contexts
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/sshd,enabled,0.7,sshd support : parser and brute-force detection,collections

Acquisition config

Stock config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # paste output here

On Windows:

C:> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml

paste output here

Config show

$ cscli config show
# paste output here

Prometheus metrics

$ cscli metrics
# paste output here

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

nvtkaszpir avatar Jun 22 '25 14:06 nvtkaszpir

@nvtkaszpir: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

github-actions[bot] avatar Jun 22 '25 14:06 github-actions[bot]

it works if I specify full path in dev.yaml in config_paths.pattern_dir:

common:
  log_media: stdout
  log_level: info
config_paths:
  config_dir: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config
  data_dir: /home/kaszpir/src/crowdsec-v1.6.9/tests/./data/
  notification_dir: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/notifications/
  plugin_dir: /home/kaszpir/src/crowdsec-v1.6.9/tests/./plugins/
  pattern_dir: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/patterns/
  #simulation_path: /etc/crowdsec/config/simulation.yaml
  #hub_dir: /etc/crowdsec/hub/
  #index_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/hub/.index.json
crowdsec_service:
  acquisition_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/acquis.yaml
  parser_routines: 1
plugin_config:
  user: kaszpir # plugin process would be ran on behalf of this user
  group: kaszpir # plugin process would be ran on behalf of this group
cscli:
  output: human
db_config:
  type: sqlite
  db_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./data/crowdsec.db
  user: root
  password: crowdsec
  db_name: crowdsec
  host: "172.17.0.2"
  port: 3306
  flush:
    #max_items: 10000
    #max_age: 168h
api:
  client:
    credentials_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/local_api_credentials.yaml
  server:
    console_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/console.yaml
    #insecure_skip_verify: true
    listen_uri: 127.0.0.1:8081
    profiles_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/profiles.yaml
    tls:
      #cert_file: /home/kaszpir/src/crowdsec-v1.6.9/tests/./cert.pem
      #key_file: /home/kaszpir/src/crowdsec-v1.6.9/tests/./key.pem
    online_client: # Central API
      credentials_path: /home/kaszpir/src/crowdsec-v1.6.9/tests/./config/online_api_credentials.yaml
prometheus:
  enabled: true
  level: full

when using relative path such as ./patterns/ it does not work

nvtkaszpir avatar Jun 22 '25 18:06 nvtkaszpir

Yes we really need to update these instructions as outlined in #3183

LaurenceJJones avatar Jun 23 '25 07:06 LaurenceJJones

From v1.7, this will be displayed if a relative directory is specified in the configuration file

WARN[2025-08-26T15:39:42+02:00] Using a relative path for "pattern" is deprecated and will be disallowed in a future release

The same config.yaml is used by cscli and the crowdsec daemon, it was never clear if the pathnames would be relative to the current directory or to the configuration directory.

mmetc avatar Aug 26 '25 13:08 mmetc