False positive http-crawl-non_statics Jellyseerr

Open jalapeno1083 opened this issue 1 year ago • 3 comments

Describe the bug False positive when scrolling through Jellyseerr and loading a few different pages and scrolling down. This makes the client load many thumbnails.

edit: I just double checked. It ~~only~~ happens when I scroll down my request list domain.com/requests edit2: Ok it also happens, when browsing normally. Scrolling down the request list just gets you banned faster.

To Reproduce Install Jellyseerr and start browsing and scrolling. To get yourself banned even faster, fill your requests list and then scroll down the requests page domain.com/requests

Expected behavior

Using Jellyseerr normally like browsing, scrolling, loading thumbnails, and scrolling down the requests lists on Jellyseerr without getting banned.

Info about alert

cscli alerts inspect

################################################################################################

ID : 154
Date : 2024-09-29T14:30:09Z
Machine : localhost
Simulation : false
Remediation : true
Reason : crowdsecurity/http-crawl-non_statics
Events Count : 74
Scope:Value : Ip:XXXXX
Country : XXXXX
AS : XXXXX
Begin : 2024-09-29 14:29:50.845089625 +0000 UTC
End : 2024-09-29 14:30:08.794147427 +0000 UTC
UUID : 76bc859b-7c5d-4eb7-b829-c1eb5a1c6594
Context : +------------+---------------------+ | Key | Value | +------------+---------------------+ | method | GET | | status | 304 | | target_uri | /api/v1/request/585 | | target_uri | /api/v1/request/621 | | target_uri | /api/v1/request/642 | | target_uri | /api/v1/request/630 | | target_uri | /api/v1/request/633 | | target_uri | /api/v1/request/599 | | user_agent | - | +------------+---------------------+
Events :
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/585 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/621 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/642 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/630 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/633 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+
Date: 2024-09-29 14:30:07 +0000 UTC +---------------------+-----------------------------+ | Key | Value | +---------------------+-----------------------------+ | ASNNumber | XXXXX | | ASNOrg | XXXXX | | IsInEU | false | | IsoCode | XXXXX | | SourceRange | XXXXX | | datasource_path | /var/log/traefik/access.log | | datasource_type | file | | http_args_len | 0 | | http_path | /api/v1/request/599 | | http_status | 304 | | http_user_agent | - | | http_verb | GET | | log_type | http_access-log | | service | http | | source_ip | XXXXX | | timestamp | 2024-09-29T14:30:07Z | | traefik_router_name | jellyseerr@file | | user | - | +---------------------+-----------------------------+

Additional context

Collections in use

COLLECTIONS

Name 📦 Status Version Local Path

crowdsecurity/base-http-scenarios ✔️ enabled 1.0 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/http-cve ✔️ enabled 2.7 /etc/crowdsec/collections/http-cve.yaml
crowdsecurity/iptables ✔️ enabled 0.2 /etc/crowdsec/collections/iptables.yaml
crowdsecurity/linux ✔️ enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/nginx ✔️ enabled 0.2 /etc/crowdsec/collections/nginx.yaml
crowdsecurity/sshd ✔️ enabled 0.5 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/traefik ✔️ enabled 0.1 /etc/crowdsec/collections/traefik.yaml
crowdsecurity/whitelist-good-actors ✔️ enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml LePresidente/jellyfin ✔️ enabled 0.2 /etc/crowdsec/collections/jellyfin.yml
LePresidente/jellyseerr ✔️ enabled 0.1 /etc/crowdsec/collections/jellyseerr.yml

Happy to provide any additional logs.

Sep 29 '24 15:09 jalapeno1083

I'm using the following whitelist for Jellyseerr:

name: overseerr-jellyseerr-whitelist
description: "Whitelist events from Overseerr and Jellyseerr"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Overseerr/Jellyseerr whitelist"
  expression:
   - evt.Meta.http_status in ['200', '499'] && evt.Parsed.static_ressource == 'false' && evt.Meta.http_verb == 'GET' && evt.Meta.http_path matches '\\/api\\/v1\\/(movie|tv|request)\\/(\\d+)' # When browsing Movies, Series or Requests

Put it into /etc/crowdsec/config/parsers/s02-enrich. Maybe someone could merge it with the Jellyseerr and the Overseerr collections?

Oct 06 '24 18:10 DanteMS

Thanks for the information. I've just recently encountered this issue.

My set up had been working with the acquisition template looking directly at /var/log/jellyseerr/overseerr-*.log. After getting banned (http status 403 then 200) I have updated the acquisition file per the hub example and added a whitelist for Jellyseerr. The regex expression above didn't work for me and I've found only whitlisting http_status 200 to have been sufficient (so far). My whitelist for Jellyseerr on traefik:

name: crowdsecurity/jellyseerr-whitelists
description: "Whitelist false positives from Jellyseerr api"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Whitelist false positive from Jellyseerr api"
  expression:
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/movie/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/tv/'
   - evt.Parsed.traefik_router_name == 'jellyseerr@docker' && evt.Meta.http_verb == 'GET' && evt.Meta.http_status == '200' && evt.Parsed.request contains '/api/v1/request/'

Oct 22 '24 01:10 wacomoto

I was having the same issue with Jellyseerr. Scrolling fast on the requests page would reliably cause http-crawl-non_statics to trigger. I am using nginx and having crowdsec parse nginx's access and error logs. The whitelist from @DanteMS works and I saw the whitelist count go up but I was still getting false positives. Investigating nginx's access log, I noticed Jellyseerr was also returning an HTTP status code 304 for a decent amount of requests which are not caught by the whitelist. Requests were also returned with the normal 200 and some with 499. After changing the whitelist to also include the HTTP 304 status code, all false positives stopped as all the requests are now captured in the whitelist.

name: overseerr-jellyseerr-whitelist
description: "Whitelist events from Overseerr and Jellyseerr"
filter: "evt.Meta.service == 'http' && evt.Meta.log_type in ['http_access-log', 'http_error-log']"
whitelist:
  reason: "Overseerr/Jellyseerr whitelist"
  expression:
   - evt.Meta.http_status in ['200', '304', '499'] && evt.Parsed.static_ressource == 'false' && evt.Meta.http_verb == 'GET' && evt.Meta.http_path matches '\\/api\\/v1\\/(movie|tv|request)\\/(\\d+)' # When browsing Movies, Series or Requests

Jan 27 '25 03:01 Jgigantino31