crowdsec
crowdsec copied to clipboard
crowdsecurity
Crowdsec memory usage grows until OOMKilled
What happened?
I have Crowdsec running on a somewhat memory limited environment, where more than 1GB usage causes it to get OOMKilled. As far I as I understand, Crowdsec should be able to run with less memory. The memory usage over the last few days looks like this:
Here is an attachment with the output from the debug/pprof:
What did you expect to happen?
Memory usage to stay at a constant value.
How can we reproduce it (as minimally and precisely as possible)?
The pprof graph seems to indicate this might be something related to TLS settings, so it might be that enabling them in a certain way might reproduce this issue.
Anything else we need to know?
No response
Crowdsec version
$ cscli version
version: v1.6.8-debian-pragmatic-amd64-f209766e
Codename: alphaga
BuildDate: 2025-03-25_14:51:10
GoVersion: 1.24.1
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.8-debian-pragmatic-amd64-f209766e-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog
OS version
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux debian-networking 6.1.0-34-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.135-1 (2025-04-25) x86_64 GNU/Linux
Enabled collections and parsers
$ cscli hub list -o raw
Loaded: 141 parsers, 10 postoverflows, 764 scenarios, 10 contexts, 4 appsec-configs, 112 appsec-rules, 144 collections
Unmanaged items: 62 local, 0 tainted
name,status,version,description,type
crowdsecurity/appsec-logs,enabled,0.5,Parse Appsec events,parsers
crowdsecurity/dateparse-enrich,"enabled,local",,,parsers
crowdsecurity/geoip-enrich,"enabled,local",,,parsers
crowdsecurity/http-logs,"enabled,local",,,parsers
crowdsecurity/nginx-logs,"enabled,local",,,parsers
crowdsecurity/sshd-logs,"enabled,local",,,parsers
crowdsecurity/syslog-logs,"enabled,local",,,parsers
crowdsecurity/whitelists,"enabled,local",,,parsers
crowdsecurity/apache_log4j2_cve-2021-44228,"enabled,local",,,scenarios
crowdsecurity/appsec-native,enabled,0.1,Identify attacks flagged by CrowdSec AppSec via native rules,scenarios
crowdsecurity/appsec-vpatch,enabled,0.6,Identify attacks flagged by CrowdSec AppSec,scenarios
crowdsecurity/CVE-2017-9841,"enabled,local",,,scenarios
crowdsecurity/CVE-2019-18935,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-26134,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-35914,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-37042,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-41082,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-41697,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-42889,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-44877,"enabled,local",,,scenarios
crowdsecurity/CVE-2022-46169-bf,"enabled,local",,,scenarios
crowdsecurity/CVE-2023-22515,"enabled,local",,,scenarios
crowdsecurity/CVE-2023-22518,"enabled,local",,,scenarios
crowdsecurity/CVE-2023-49103,"enabled,local",,,scenarios
crowdsecurity/CVE-2024-0012,"enabled,local",,,scenarios
crowdsecurity/CVE-2024-38475,"enabled,local",,,scenarios
crowdsecurity/CVE-2024-9474,"enabled,local",,,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,"enabled,local",,,scenarios
crowdsecurity/fortinet-cve-2018-13379,"enabled,local",,,scenarios
crowdsecurity/fortinet-cve-2022-40684,"enabled,local",,,scenarios
crowdsecurity/grafana-cve-2021-43798,"enabled,local",,,scenarios
crowdsecurity/http-admin-interface-probing,"enabled,local",,,scenarios
crowdsecurity/http-backdoors-attempts,"enabled,local",,,scenarios
crowdsecurity/http-bad-user-agent,"enabled,local",,,scenarios
crowdsecurity/http-crawl-non_statics,"enabled,local",,,scenarios
crowdsecurity/http-cve-2021-41773,"enabled,local",,,scenarios
crowdsecurity/http-cve-2021-42013,"enabled,local",,,scenarios
crowdsecurity/http-cve-probing,"enabled,local",,,scenarios
crowdsecurity/http-generic-bf,"enabled,local",,,scenarios
crowdsecurity/http-open-proxy,"enabled,local",,,scenarios
crowdsecurity/http-path-traversal-probing,"enabled,local",,,scenarios
crowdsecurity/http-probing,"enabled,local",,,scenarios
crowdsecurity/http-sensitive-files,"enabled,local",,,scenarios
crowdsecurity/http-sqli-probbing-detection,"enabled,local",,,scenarios
crowdsecurity/http-wordpress-scan,"enabled,local",,,scenarios
crowdsecurity/http-xss-probbing,"enabled,local",,,scenarios
crowdsecurity/jira_cve-2021-26086,"enabled,local",,,scenarios
crowdsecurity/netgear_rce,"enabled,local",,,scenarios
crowdsecurity/nginx-req-limit-exceeded,"enabled,local",,,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,"enabled,local",,,scenarios
crowdsecurity/spring4shell_cve-2022-22965,"enabled,local",,,scenarios
crowdsecurity/ssh-bf,"enabled,local",,,scenarios
crowdsecurity/ssh-cve-2024-6387,"enabled,local",,,scenarios
crowdsecurity/ssh-refused-conn,enabled,0.1,Detect sshd refused connections,scenarios
crowdsecurity/ssh-slow-bf,"enabled,local",,,scenarios
crowdsecurity/thinkphp-cve-2018-20062,"enabled,local",,,scenarios
crowdsecurity/vmware-cve-2022-22954,"enabled,local",,,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,"enabled,local",,,scenarios
ltsich/http-w00tw00t,"enabled,local",,,scenarios
bf_base.yaml,"enabled,local",,,contexts
crowdsecurity/appsec_base,enabled,0.9,,contexts
http_base.yaml,"enabled,local",,,contexts
crowdsecurity/appsec-default,enabled,0.2,,appsec-configs
crowdsecurity/generic-rules,enabled,0.3,,appsec-configs
crowdsecurity/virtual-patching,enabled,0.4,,appsec-configs
crowdsecurity/base-config,enabled,0.1,,appsec-rules
crowdsecurity/generic-freemarker-ssti,enabled,0.3,Generic FreeMarker SSTI,appsec-rules
crowdsecurity/generic-wordpress-uploads-php,enabled,0.1,Detect php execution in wordpress uploads directory,appsec-rules
crowdsecurity/vpatch-connectwise-auth-bypass,enabled,0.3,Detect exploitation of auth bypass in ConnectWise ScreenConnect,appsec-rules
crowdsecurity/vpatch-CVE-2002-1131,enabled,0.1,"Detects XSS attempts in SquirrelMail 1.2.6/1.2.7 via unsanitized input in addressbook, options, search, and help modules.",appsec-rules
crowdsecurity/vpatch-CVE-2007-0885,enabled,0.1,Detects XSS vulnerability in Jira Rainbow.Zen via the id parameter in BrowseProject.jspa.,appsec-rules
crowdsecurity/vpatch-CVE-2017-9841,enabled,0.3,PHPUnit RCE (CVE-2017-9841),appsec-rules
crowdsecurity/vpatch-CVE-2018-1000861,enabled,0.1,Jenkins - RCE (CVE-2018-1000861),appsec-rules
crowdsecurity/vpatch-CVE-2018-10562,enabled,0.2,Dasan GPON RCE (CVE-2018-10562),appsec-rules
crowdsecurity/vpatch-CVE-2018-13379,enabled,0.2,Fortinet FortiOS - Credentials Disclosure (CVE-2018-13379),appsec-rules
crowdsecurity/vpatch-CVE-2018-20062,enabled,0.1,ThinkPHP - RCE (CVE-2018-20062),appsec-rules
crowdsecurity/vpatch-CVE-2019-1003030,enabled,0.1,Jenkins - RCE (CVE-2019-1003030),appsec-rules
crowdsecurity/vpatch-CVE-2019-12989,enabled,0.3,Citrix SQLi (CVE-2019-12989),appsec-rules
crowdsecurity/vpatch-CVE-2019-18935,enabled,0.1,Telerik - RCE (CVE-2019-18935),appsec-rules
crowdsecurity/vpatch-CVE-2020-11738,enabled,0.6,Wordpress Snap Creek Duplicator - Path Traversal (CVE-2020-11738),appsec-rules
crowdsecurity/vpatch-CVE-2020-17496,enabled,0.1,vBulletin RCE (CVE-2020-17496),appsec-rules
crowdsecurity/vpatch-CVE-2020-5902,enabled,0.1,F5 BIG-IP TMUI - RCE (CVE-2020-5902),appsec-rules
crowdsecurity/vpatch-CVE-2020-9054,enabled,0.1,Detects pre-authentication command injection in Zyxel NAS devices via weblogin.cgi,appsec-rules
crowdsecurity/vpatch-CVE-2021-22941,enabled,0.3,Citrix RCE (CVE-2021-22941),appsec-rules
crowdsecurity/vpatch-CVE-2021-26086,enabled,0.1,Atlassian Jira Server/Data Center 8.4.0 - Limited Remote File Read/Include (CVE-2021-26086),appsec-rules
crowdsecurity/vpatch-CVE-2021-26294,enabled,0.1,Detects unauthorized access to AfterLogic Aurora/WebMail Pro WebDAV endpoint using default caldav_public_user credentials and path traversal.,appsec-rules
crowdsecurity/vpatch-CVE-2021-3129,enabled,0.4,Laravel with Ignition Debug Mode RCE (CVE-2021-3129),appsec-rules
crowdsecurity/vpatch-CVE-2021-43798,enabled,0.3,Grafana Arbitrary File Read (CVE-2021-43798),appsec-rules
crowdsecurity/vpatch-CVE-2021-44529,enabled,0.2,Detects code injection in Ivanti EPM CSA via cookie manipulation (CVE-2021-44529),appsec-rules
crowdsecurity/vpatch-CVE-2022-1388,enabled,0.1,Detects F5 BIG-IP iControl REST authentication bypass and RCE via crafted POST to /mgmt/tm/util/bash with X-F5-Auth-Token header.,appsec-rules
crowdsecurity/vpatch-CVE-2022-22954,enabled,0.2,VMWare Workspace ONE Access RCE (CVE-2022-22954),appsec-rules
crowdsecurity/vpatch-CVE-2022-22965,enabled,0.2,Spring4Shell - RCE (CVE-2022-22965),appsec-rules
crowdsecurity/vpatch-CVE-2022-25488,enabled,0.4,Atom CMS - SQLi (CVE-2022-25488),appsec-rules
crowdsecurity/vpatch-CVE-2022-26134,enabled,0.2,Confluence - RCE (CVE-2022-26134),appsec-rules
crowdsecurity/vpatch-CVE-2022-27926,enabled,0.4,Zimbra Collaboration XSS (CVE-2022-27926),appsec-rules
crowdsecurity/vpatch-CVE-2022-35914,enabled,0.5,GLPI RCE (CVE-2022-35914),appsec-rules
crowdsecurity/vpatch-CVE-2022-41082,enabled,0.1,Microsoft Exchange - RCE (CVE-2022-41082),appsec-rules
crowdsecurity/vpatch-CVE-2022-44877,enabled,0.2,CentOS Web Panel 7 RCE (CVE-2022-44877),appsec-rules
crowdsecurity/vpatch-CVE-2022-46169,enabled,0.5,Cacti RCE (CVE-2022-46169),appsec-rules
crowdsecurity/vpatch-CVE-2023-0297,enabled,0.1,"Detects pre-auth remote code execution in PyLoad via code injection in the ""jk"" parameter of /flash/addcrypted2.",appsec-rules
crowdsecurity/vpatch-CVE-2023-1389,enabled,0.1,TP-Link Archer AX21 - RCE (CVE-2023-1389),appsec-rules
crowdsecurity/vpatch-CVE-2023-20198,enabled,0.6,CISCO IOS XE Account Creation (CVE-2023-20198),appsec-rules
crowdsecurity/vpatch-CVE-2023-22515,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-22515),appsec-rules
crowdsecurity/vpatch-CVE-2023-22527,enabled,0.2,RCE using SSTI in Confluence (CVE-2023-22527),appsec-rules
crowdsecurity/vpatch-CVE-2023-23752,enabled,0.1,Joomla! Webservice - Password Disclosure (CVE-2023-23752),appsec-rules
crowdsecurity/vpatch-CVE-2023-24489,enabled,0.2,Citrix ShareFile RCE (CVE-2023-24489),appsec-rules
crowdsecurity/vpatch-CVE-2023-28121,enabled,0.1,WooCommerce auth bypass (CVE-2023-28121),appsec-rules
crowdsecurity/vpatch-CVE-2023-33617,enabled,0.4,Atlassian Confluence Privesc (CVE-2023-33617),appsec-rules
crowdsecurity/vpatch-CVE-2023-34362,enabled,0.6,MOVEit Transfer RCE (CVE-2023-34362),appsec-rules
crowdsecurity/vpatch-CVE-2023-35078,enabled,0.1,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35078),appsec-rules
crowdsecurity/vpatch-CVE-2023-35082,enabled,0.2,MobileIron Core Remote Unauthenticated API Access (CVE-2023-35082),appsec-rules
crowdsecurity/vpatch-CVE-2023-3519,enabled,0.3,Citrix RCE (CVE-2023-3519),appsec-rules
crowdsecurity/vpatch-CVE-2023-38205,enabled,0.3,Adobe ColdFusion Access Control Bypass (CVE-2023-38205),appsec-rules
crowdsecurity/vpatch-CVE-2023-40044,enabled,0.3,WS_FTP .NET deserialize RCE (CVE-2023-40044),appsec-rules
crowdsecurity/vpatch-CVE-2023-42793,enabled,0.3,JetBrains Teamcity Auth Bypass (CVE-2023-42793),appsec-rules
crowdsecurity/vpatch-CVE-2023-46805,enabled,0.4,Ivanti Connect Auth Bypass (CVE-2023-46805),appsec-rules
crowdsecurity/vpatch-CVE-2023-47218,enabled,0.2,QNAP QTS - RCE (CVE-2023-47218),appsec-rules
crowdsecurity/vpatch-CVE-2023-49070,enabled,0.1,Apache OFBiz - RCE (CVE-2023-49070),appsec-rules
crowdsecurity/vpatch-CVE-2023-50164,enabled,0.6,Apache Struts2 Path Traversal (CVE-2023-50164),appsec-rules
crowdsecurity/vpatch-CVE-2023-6553,enabled,0.1,Backup Migration plugin for WordPress RCE (CVE-2023-6553),appsec-rules
crowdsecurity/vpatch-CVE-2023-7028,enabled,0.2,Gitlab Password Reset Account Takeover (CVE-2023-7028),appsec-rules
crowdsecurity/vpatch-CVE-2024-0012,enabled,0.1,PanOS - Authentication Bypass (CVE-2024-0012),appsec-rules
crowdsecurity/vpatch-CVE-2024-1212,enabled,0.3,Progress Kemp LoadMaster Unauthenticated Command Injection (CVE-2024-1212),appsec-rules
crowdsecurity/vpatch-CVE-2024-22024,enabled,0.1,Ivanti Connect Secure - XXE (CVE-2024-22024),appsec-rules
crowdsecurity/vpatch-CVE-2024-23897,enabled,0.4,Jenkins CLI RCE (CVE-2024-23897),appsec-rules
crowdsecurity/vpatch-CVE-2024-27198,enabled,0.5,Teamcity - Authentication Bypass (CVE-2024-27198),appsec-rules
crowdsecurity/vpatch-CVE-2024-27292,enabled,0.2,Local File Inclusion - Docassemble,appsec-rules
crowdsecurity/vpatch-CVE-2024-27348,enabled,0.1,Apache HugeGraph-Server - RCE (CVE-2024-27348),appsec-rules
crowdsecurity/vpatch-CVE-2024-27564,enabled,0.3,Detects SSRF attack via pictureproxy.php in ChatGPT application,appsec-rules
crowdsecurity/vpatch-CVE-2024-27954,enabled,0.1,WP Automatic - Path Traversal (CVE-2024-27954),appsec-rules
crowdsecurity/vpatch-CVE-2024-27956,enabled,0.1,WordPress Automatic Plugin - SQLi (CVE-2024-27956),appsec-rules
crowdsecurity/vpatch-CVE-2024-28255,enabled,0.1,OpenMetadata - Authentication Bypass (CVE-2024-28255),appsec-rules
crowdsecurity/vpatch-CVE-2024-28987,enabled,0.1,SolarWinds WHD Hardcoded Credentials (CVE-2024-28987),appsec-rules
crowdsecurity/vpatch-CVE-2024-29824,enabled,0.1,Ivanti EPM - SQLi (CVE-2024-29824),appsec-rules
crowdsecurity/vpatch-CVE-2024-29849,enabled,0.5,Veeam Backup Enterprise Manager - Authentication Bypass (CVE-2024-29849),appsec-rules
crowdsecurity/vpatch-CVE-2024-29973,enabled,0.1,Zyxel - RCE (CVE-2024-29973),appsec-rules
crowdsecurity/vpatch-CVE-2024-32113,enabled,0.1,Apache OFBiz - Path Traversal (CVE-2024-32113),appsec-rules
crowdsecurity/vpatch-CVE-2024-3272,enabled,0.1,D-Link NAS - RCE (CVE-2024-3272),appsec-rules
crowdsecurity/vpatch-CVE-2024-3273,enabled,0.1,D-LINK NAS Command Injection (CVE-2024-3273),appsec-rules
crowdsecurity/vpatch-CVE-2024-32870,enabled,0.1,Detects unauthorized access to iTop Hub Connector information disclosure endpoint.,appsec-rules
crowdsecurity/vpatch-CVE-2024-34102,enabled,0.1,Adobe Commerce & Magento - XXE (CVE-2024-34102),appsec-rules
crowdsecurity/vpatch-CVE-2024-38816,enabled,0.2,Spring - Path Traversal (CVE-2024-38816),appsec-rules
crowdsecurity/vpatch-CVE-2024-38856,enabled,0.1,Apache OFBiz Incorrect Authorization (CVE-2024-38856),appsec-rules
crowdsecurity/vpatch-CVE-2024-41713,enabled,0.2,Mitel MiCollab - Path Traversal (CVE-2024-41713),appsec-rules
crowdsecurity/vpatch-CVE-2024-4577,enabled,0.1,PHP CGI Command Injection - CVE-2024-4577,appsec-rules
crowdsecurity/vpatch-CVE-2024-51378,enabled,0.1,Cyberpanel - RCE (CVE-2024-51378),appsec-rules
crowdsecurity/vpatch-CVE-2024-51567,enabled,0.1,CyberPanel RCE (CVE-2024-51567),appsec-rules
crowdsecurity/vpatch-CVE-2024-52301,enabled,0.1,Laravel - Parameter Injection (CVE-2024-52301),appsec-rules
crowdsecurity/vpatch-CVE-2024-57727,enabled,0.4,Detects unauthenticated path traversal attempts targeting SimpleHelp <= 5.5.7,appsec-rules
crowdsecurity/vpatch-CVE-2024-6205,enabled,0.2,PayPlus Payment Gateway WordPress plugin - SQL Injection (CVE-2024-6205),appsec-rules
crowdsecurity/vpatch-CVE-2024-7593,enabled,0.1,Ivanti vTM - Authentication Bypass (CVE-2024-7593),appsec-rules
crowdsecurity/vpatch-CVE-2024-8190,enabled,0.1,Ivanti Cloud Services Appliance - RCE (CVE-2024-8190),appsec-rules
crowdsecurity/vpatch-CVE-2024-8963,enabled,0.2,Ivanti CSA - Path Traversal (CVE-2024-8963),appsec-rules
crowdsecurity/vpatch-CVE-2024-9465,enabled,0.2,Palo Alto Expedition - SQL Injection (CVE-2024-9465),appsec-rules
crowdsecurity/vpatch-CVE-2024-9474,enabled,0.3,PanOS - Privilege Escalation (CVE-2024-9474),appsec-rules
crowdsecurity/vpatch-CVE-2025-24893,enabled,0.2,Detects arbitrary remote code execution vulnerability in XWiki via SolrSearch.,appsec-rules
crowdsecurity/vpatch-CVE-2025-28367,enabled,0.1,Detects directory traversal in mojoPortal BetterImageGallery API Controller (CVE-2025-28367),appsec-rules
crowdsecurity/vpatch-CVE-2025-29927,enabled,0.1,Next.js Middleware Bypass - (CVE-2025-29927),appsec-rules
crowdsecurity/vpatch-CVE-2025-31161,enabled,0.1,Detects authentication bypass in CrushFTP via crafted Authorization header and specific endpoint access.,appsec-rules
crowdsecurity/vpatch-CVE-2025-31324,enabled,0.1,SAP NetWeaver - File Upload (CVE-2025-31324),appsec-rules
crowdsecurity/vpatch-CVE-2025-3248,enabled,0.1,Detects unauthenticated remote code execution in Langflow via /api/v1/validate/code endpoint.,appsec-rules
crowdsecurity/vpatch-env-access,enabled,0.1,Detect access to .env files,appsec-rules
crowdsecurity/vpatch-git-config,enabled,0.2,Detect access to .git files,appsec-rules
crowdsecurity/vpatch-laravel-debug-mode,enabled,0.3,Detect bots exploiting laravel debug mode,appsec-rules
crowdsecurity/vpatch-symfony-profiler,enabled,0.1,Detect abuse of symfony profiler,appsec-rules
base-http-scenarios.yaml,"enabled,local",,,collections
crowdsecurity/appsec-generic-rules,enabled,0.7,A collection of generic attack vectors for additional protection.,collections
crowdsecurity/appsec-virtual-patching,enabled,7.0,"a generic virtual patching collection, suitable for most web servers.",collections
http-cve.yaml,"enabled,local",,,collections
linux.yaml,"enabled,local",,,collections
nginx.yaml,"enabled,local",,,collections
sshd.yaml,"enabled,local",,,collections
Acquisition config
```console
# On Linux:
$ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/*
---
filenames:
- /var/log/nginx/*.log
- ./tests/nginx/nginx.log
labels:
type: nginx
filenames:
- /var/log/auth.log
- /var/log/syslog labels: type: syslog
source: journalctl journalctl_filter:
- _SYSTEMD_UNIT=ssh.service labels: type: syslog
filename: /var/log/apache2/*.log labels: type: apache2
listen_addr: 0.0.0.0:7422 appsec_config: crowdsecurity/appsec-default name: AppSecComponent source: appsec labels: type: appsec
Config show
$ cscli config show
Global:
- Configuration Folder : /etc/crowdsec
- Data Folder : /var/lib/crowdsec/data
- Hub Folder : /var/lib/crowdsec/hub
- Simulation File : /etc/crowdsec/simulation.yaml
- Log Folder : /var/log
- Log level : info
- Log Media : file
Crowdsec:
- Acquisition File : /etc/crowdsec/acquis.yaml
- Parsers routines : 1
- Acquisition Folder : /etc/crowdsec/acquis.d
cscli:
- Output : human
- Hub Branch :
API Client:
- URL : https://debian-crowdsec:8080/
- Login :
- Credentials File : /etc/crowdsec/local_api_credentials.yaml
Local API Server:
- Listen URL : x:8080
- Listen Socket :
- Profile File : /etc/crowdsec/profiles.yaml
- Cert File : /etc/ssl/crowdsec.crt
- Key File : /etc/ssl/crowdsec.key
- CA Cert : /usr/local/share/ca-certificates/debian-crowdsec.crt
- Allowed Agents OU : Crowdsec
- Allowed Bouncers OU : Crowdsec
- Trusted IPs:
- 127.0.0.1
- ::1
- 10.0.0.0/24
- Database:
- Type : sqlite
- Path : /var/lib/crowdsec/data/crowdsec.db
- Flush age : 7d
- Flush size : 5000
Prometheus metrics
$ cscli metrics
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Acquisition Metrics │
├───────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├───────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/nginx/access.log │ 22.43k │ 22.43k │ - │ 21.74k │ - │
│ file:/var/log/nginx/error.log │ 34 │ - │ 34 │ - │ - │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=ssh.service │ 672 │ - │ 672 │ - │ - │
╰───────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
╭─────────────────────────────────────────────────────╮
│ Local API Alerts │
├─────────────────────────────────────────────┬───────┤
│ Reason │ Count │
├─────────────────────────────────────────────┼───────┤
│ crowdsecurity/CVE-2022-41082 │ 2 │
│ crowdsecurity/generic-wordpress-uploads-php │ 1 │
│ crowdsecurity/http-bad-user-agent │ 16 │
│ crowdsecurity/http-crawl-non_statics │ 1 │
│ crowdsecurity/http-cve-2021-42013 │ 9 │
│ crowdsecurity/http-sensitive-files │ 2 │
│ crowdsecurity/CVE-2017-9841 │ 11 │
│ crowdsecurity/http-cve-2021-41773 │ 11 │
│ crowdsecurity/http-open-proxy │ 8 │
│ crowdsecurity/http-probing │ 5 │
│ crowdsecurity/vpatch-git-config │ 4 │
╰─────────────────────────────────────────────┴───────╯
╭───────────────────────────────────────╮
│ Appsec Metrics │
├─────────────────┬───────────┬─────────┤
│ Appsec Engine │ Processed │ Blocked │
├─────────────────┼───────────┼─────────┤
│ AppSecComponent │ 105.20k │ - │
╰─────────────────┴───────────┴─────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x │
│ ) since 2025-05-12 │
│ 07:26:03 +0000 UT │
│ C │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 42.11k │
╰────────┴───────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x │
│ ) since 2025-05-12 │
│ 07:25:24 +0000 UT │
│ C │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 34.45k │
╰────────┴───────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x) │
│ since 2025-05-12 │
│ 07:25:24 +0000 UTC │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 42.41k │
╰────────┴───────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x │
│ ) since 2025- │
│ 05-12 07:25:34 +00 │
│ 00 UTC │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 30.15k │
╰────────┴───────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x │
│ ) since 2025-05 │
│ -12 07:24:54 +0000 │
│ UTC │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 79.98k │
╰────────┴───────────╯
╭────────────────────╮
│ Bouncer Metrics (c │
│ rowdsec-nginx-boun │
│ cer-x │
│ ) since 2025-05 │
│ -12 07:25:03 +0000 │
│ UTC │
├────────┬───────────┤
│ Origin │ processed │
│ │ request │
├────────┼───────────┤
│ Total │ 20.13k │
╰────────┴───────────╯
╭──────────────────────────────────────────────────────────────────╮
│ Local API Decisions │
├──────────────────────────────────────┬──────────┬────────┬───────┤
│ Reason │ Origin │ Action │ Count │
├──────────────────────────────────────┼──────────┼────────┼───────┤
│ http:exploit │ CAPI │ ban │ 173 │
│ crowdsecurity/http-bad-user-agent │ crowdsec │ ban │ 7 │
│ crowdsecurity/http-crawl-non_statics │ crowdsec │ ban │ 1 │
│ crowdsecurity/http-cve-2021-41773 │ crowdsec │ ban │ 3 │
│ crowdsecurity/http-cve-2021-42013 │ crowdsec │ ban │ 2 │
│ crowdsecurity/http-probing │ crowdsec │ ban │ 2 │
│ http:crawl │ CAPI │ ban │ 59 │
│ http:scan │ CAPI │ ban │ 14759 │
│ crowdsecurity/CVE-2017-9841 │ crowdsec │ ban │ 8 │
│ crowdsecurity/CVE-2022-41082 │ crowdsec │ ban │ 1 │
│ crowdsecurity/http-sensitive-files │ crowdsec │ ban │ 2 │
╰──────────────────────────────────────┴──────────┴────────┴───────╯
╭───────────────────────────────────────╮
│ Local API Metrics │
├──────────────────────┬────────┬───────┤
│ Route │ Method │ Hits │
├──────────────────────┼────────┼───────┤
│ /v1/alerts │ POST │ 26 │
│ /v1/allowlists │ GET │ 2902 │
│ /v1/decisions │ GET │ 94553 │
│ /v1/decisions/stream │ HEAD │ 22363 │
│ /v1/heartbeat │ GET │ 23207 │
│ /v1/usage-metrics │ POST │ 2872 │
│ /v1/watchers/login │ POST │ 400 │
╰──────────────────────┴────────┴───────╯
╭────────────────────────────────────────────────────────────────────────────────────╮
│ Local API Bouncers Metrics │
├────────────────────────────────────────────┬──────────────────────┬────────┬───────┤
│ Bouncer │ Route │ Method │ Hits │
├────────────────────────────────────────────┼──────────────────────┼────────┼───────┤
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 12088 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2886 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 9071 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2884 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 12232 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2888 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 26422 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 3812 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 8690 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2887 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 5802 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2008 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 14449 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2943 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions │ GET │ 5799 │
│ crowdsec-nginx-bouncer-x │ /v1/decisions/stream │ HEAD │ 2055 │
╰────────────────────────────────────────────┴──────────────────────┴────────┴───────╯
╭────────────────────────────────────────────────────────────────────────────────╮
│ Local API Bouncers Decisions │
├────────────────────────────────────────────┬───────────────┬───────────────────┤
│ Bouncer │ Empty answers │ Non-empty answers │
├────────────────────────────────────────────┼───────────────┼───────────────────┤
│ crowdsec-nginx-bouncer-x │ 9071 │ 0 │
│ crowdsec-nginx-bouncer-x │ 8690 │ 0 │
│ crowdsec-nginx-bouncer-x │ 5802 │ 0 │
│ crowdsec-nginx-bouncer-x │ 14449 │ 0 │
│ crowdsec-nginx-bouncer-x │ 5799 │ 0 │
│ crowdsec-nginx-bouncer-x │ 12232 │ 0 │
│ crowdsec-nginx-bouncer-x │ 26422 │ 0 │
│ crowdsec-nginx-bouncer-x │ 12088 │ 0 │
╰────────────────────────────────────────────┴───────────────┴───────────────────╯
╭────────────────────────────────────────────────────────────────────╮
│ Local API Machines Metrics │
├───────────────────────────────────┬────────────────┬────────┬──────┤
│ Machine │ Route │ Method │ Hits │
├───────────────────────────────────┼────────────────┼────────┼──────┤
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/allowlists │ GET │ 2902 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/alerts │ POST │ 26 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2900 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
│ DNS:x@x │ /v1/heartbeat │ GET │ 2901 │
╰───────────────────────────────────┴────────────────┴────────┴──────╯
╭──────────────────────────────────────────────────────────────╮
│ Parser Metrics │
├─────────────────────────────────┬────────┬────────┬──────────┤
│ Parsers │ Hits │ Parsed │ Unparsed │
├─────────────────────────────────┼────────┼────────┼──────────┤
│ child-crowdsecurity/http-logs │ 67.28k │ 45.26k │ 22.03k │
│ child-crowdsecurity/nginx-logs │ 22.60k │ 22.43k │ 170 │
│ child-crowdsecurity/sshd-logs │ 9.41k │ - │ 9.41k │
│ child-crowdsecurity/syslog-logs │ 672 │ 672 │ - │
│ crowdsecurity/dateparse-enrich │ 22.43k │ 22.43k │ - │
│ crowdsecurity/geoip-enrich │ 22.43k │ 22.43k │ - │
│ crowdsecurity/http-logs │ 22.43k │ 22.40k │ 29 │
│ crowdsecurity/nginx-logs │ 22.46k │ 22.43k │ 34 │
│ crowdsecurity/non-syslog │ 22.46k │ 22.46k │ - │
│ crowdsecurity/sshd-logs │ 672 │ - │ 672 │
│ crowdsecurity/syslog-logs │ 672 │ 672 │ - │
│ crowdsecurity/whitelists │ 22.43k │ 22.43k │ - │
╰─────────────────────────────────┴────────┴────────┴──────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Scenario Metrics │
├───────────────────────────────────────────┬───────────────┬───────────┬──────────────┬────────┬─────────┤
│ Scenario │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├───────────────────────────────────────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
│ crowdsecurity/CVE-2017-9841 │ - │ 4 │ 4 │ - │ - │
│ crowdsecurity/CVE-2022-41082 │ - │ 1 │ 1 │ - │ - │
│ crowdsecurity/http-backdoors-attempts │ - │ - │ 4 │ 4 │ 4 │
│ crowdsecurity/http-bad-user-agent │ - │ 13 │ 25 │ 38 │ 12 │
│ crowdsecurity/http-crawl-non_statics │ 4 │ 1 │ 20.97k │ 21.39k │ 20.97k │
│ crowdsecurity/http-cve-2021-41773 │ - │ 5 │ 5 │ - │ - │
│ crowdsecurity/http-cve-2021-42013 │ - │ 3 │ 3 │ - │ - │
│ crowdsecurity/http-open-proxy │ - │ 1 │ 1 │ - │ - │
│ crowdsecurity/http-path-traversal-probing │ - │ - │ 1 │ 1 │ 1 │
│ crowdsecurity/http-probing │ - │ 12 │ 94 │ 282 │ 82 │
│ crowdsecurity/http-sensitive-files │ - │ 1 │ 21 │ 31 │ 20 │
╰───────────────────────────────────────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯
╭──────────────────────────────────────────────────────────────────────────────╮
│ Whitelist Metrics │
├──────────────────────────┬─────────────────────────────┬───────┬─────────────┤
│ Whitelist │ Reason │ Hits │ Whitelisted │
├──────────────────────────┼─────────────────────────────┼───────┼─────────────┤
│ crowdsecurity/whitelists │ private ipv4/ipv6 ip/ranges │ 22428 │ - │
╰──────────────────────────┴─────────────────────────────┴───────┴─────────────╯
Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.
Email plugin:
type: email
name: email_default
log_level: info
timeout: 20s
format: "<html><body>\n{{range . -}}\n {{$alert := . -}}\n {{range .Decisions -}}\n
\ <p><a href=\"https://www.whois.com/whois/{{.Value}}\">{{.Value}}</a> will get
<b>{{.Type}}</b> for next <b>{{.Duration}}</b> for triggering <b>{{.Scenario}}</b>
on machine <b>{{$alert.MachineID}}</b>.</p> <p><a href=\"https://app.crowdsec.net/cti/{{.Value}}\">CrowdSec
CTI</a></p>\n {{end -}}\n{{end -}}\n</body></html>\n"
smtp_host: x
smtp_username: x
smtp_password: x
smtp_port: x
auth_type: plain
sender_name: CrowdSec
sender_email: x
email_subject: CrowdSec Notification
receiver_emails:
- x
encryption_type: starttls
@samuliy: Thanks for opening an issue, it is currently awaiting triage.
In the meantime, you can:
- Check Crowdsec Documentation to see if your issue can be self resolved.
- You can also join our Discord.
- Check Releases to make sure your agent is on the latest version.
Details
I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Could you give us some insights to the TLS settings you use, plus how many machines / remediations you use for TLS?
edit: cause all I see is nginx remediation which cant use the TLS settings. Attaching cscli {machines, bouncers} list can give us an overview you can redact any PII.
Sure! Here is my machines list:
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name IP Address Last Update Status Version OS Auth Type Last Heartbeat
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
DNS:x x 2025-05-19T08:00:42Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 50s
DNS:x x 2025-05-19T08:00:33Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 58s
DNS:x x 2025-05-19T08:00:33Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 58s
DNS:x x 2025-05-19T08:00:33Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 59s
DNS:x x 2025-05-19T08:01:32Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 0s
DNS:x x 2025-05-19T08:00:33Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 58s
DNS:x x 2025-05-19T08:00:54Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 38s
DNS:x x 2025-05-19T08:00:36Z ✔️ v1.6.8-debian-pragmatic-amd64-f209766e Debian GNU/Linux/12 tls 56s
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
And here is the bouncers list:
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name IP Address Valid Last API pull Type Version Auth Type
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:05:04Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:04:59Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:03:59Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:05:04Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:05:04Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:05:07Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:04:57Z crowdsec-nginx-bouncer v1.1.0 api-key
crowdsec-nginx-bouncer-xxxxxxxxxxxxxxxxxxx xxxxxxxxx ✔️ 2025-05-19T08:05:00Z crowdsec-nginx-bouncer v1.1.0 api-key
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
The server where the Crowdsec daemon gets oomkilled works as a central service for all these machines running nginx. The nginx machines have the local credentials set to following config:
url: https://debian-crowdsec:8080
ca_cert_path: /usr/local/share/ca-certificates/client_ca.crt
cert_path: /etc/ssl/crowdsec.crt
key_path: /etc/ssl/crowdsec.key
The nginx bouncers run with following type of config:
ENABLED=true
ENABLE_INTERNAL=true
API_URL=https://x:8080
API_KEY=x
BOUNCING_ON_TYPE=all
FALLBACK_REMEDIATION=ban
MODE=live
REQUEST_TIMEOUT=1000
CACHE_EXPIRATION=1
UPDATE_FREQUENCY=10
APPSEC_URL=http://x:7422
SSL_VERIFY=true
I have the same issue that the container goes OOM after a few days. I have the same kind of setup like the OP but then with 1 local and 1 remote firewall agent. The connection is without TLS over wireguard.
I have the same issue that the container goes OOM after a few days. I have the same kind of setup like the OP but then with 1 local and 1 remote firewall agent. The connection is without TLS over wireguard.
Then we going to need some more debug gable information, as it cant be the same as OP when the OP is using TLS and you are not.
You can dump the pprof https://docs.crowdsec.net/docs/next/observability/pprof/ stats to a file when possible also, you can use cscli support dump and this will also collect the pprof metrics for us.
Remember to include any information like custom scenarios or profiles as before when we had people complaining about OOM they were using custom scenarios that were not optimized and were collecting hundreds of events and keep in mind all events are stored in memory until overflow.
I also have that issue, but for me, it is the AppSec container that OOM's.
From my (daily rebuilt) DEV environment (from which I got the support dump as well):
Thanks for the report and the extra context.
-
AppSec can use more memory than other containers because it must read the request body before handing it to the underlying engine. This is expected for many WAF rule sets that inspect full bodies. We are evaluating improvements in this area (see #3805), but if your rules require body inspection the gains may be limited.
-
Please ensure your web server is enforcing a reasonable maximum request body size for your use case. Many servers default to around 100 MB, which is far higher than typical API traffic needs. If you do accept large uploads, consider:
- Limiting large bodies to dedicated upload paths
- Skipping or relaxing body inspection on those paths
- Using separate upstreams for upload endpoints
-
We cannot reproduce the OP reported OOM so far. To help us debug your environment, please share:
- Pprof captures (heap, profile) around the time of the spike
-
Mitigations you can try now for AppSec:
- Set a tighter max request body size on your front end (for example, NGINX
client_max_body_size 5m;if 5 MB suits your workload) - Exclude static assets and health checks from AppSec inspection
- Scope the most expensive body rules to only the endpoints that need them
- Allocate conservative memory limits and monitor usage to avoid kernel OOM kills while we analyze pprof data
- Set a tighter max request body size on your front end (for example, NGINX
With further analysis we found out increasing the chart version fixed the problem. Here are more details about it:
- Both Appsec and LAPI pods show memory usage increase over time.
- Appsec pod restarts periodically, which resets its memory usage and also frees up memory allocated by LAPI.
- This behavior is linked to constant communication between Appsec and LAPI.
- CrowdSecMode is set to
appsecwhich is mandatory for our setup. - Using
cscli support dumpa zip file can be generated containing various information including pprof heap data. - Thread about memory consumption with bouncers and appsec component enabled, caused by communication between LAPI and appsec, see here.
- Our crowdsec core version was v1.6.9.
- Our helm chart version was v0.19.4.
- Pull Request on optimizing communication: https://github.com/crowdsecurity/crowdsec/pull/3693
- Patch merged and released with crowdsec version 1.6.10, see releases
- Crowdsec version 1.6.10 is used in kubernetes helm chart version v0.19.5 (latest)
- Our helm chart was upgraded from v0.19.4 to v0.19.5.
- No more memory increase in appsec pod
Here are the memory usages of the pods in our dev environment in which we tested:
Appsec pod memory usage with v0.19.4 deployed captured on on 22.08.25:
LAPI pod memory usage with v0.19.4 deployed captured on on 22.08.25:
LAPI pod memory usage with v0.19.5 deployed on 26.08.25:
The LAPI pod memory usage is still slightly increasing, this needs to be observed over longer time frame.
Appsec pod memory usage with v0.19.5 deployed on 26.08.25:
Thank you taking the time to look into it @schnurrro
Glad to see some changes improve appsec, I am working on some other optimizations such as:
#3801 #3805 #3878
which means it will get better overtime, but some are drastically changing how the component functions.
The LAPI can periodically have spikes due to this component being the one in charge of downloading blocklists and other remote content, but >1gb is alot. Due to you have a lot of Remediation Components (bouncers)?
Yes we have three bouncers in total:
- Two linux fw-bouncers to block threats right at the entrance to our systems
- One traefik bouncer working together with appsec