actix-web-prom icon indicating copy to clipboard operation
actix-web-prom copied to clipboard

Should not extend metrics response on 404 (Uncontrolled Resource Consumption Vulnerability)

Open oscarhermoso opened this issue 1 year ago • 1 comments

Currently, when an endpoint receives a 404, it extends the Prometheus metrics response by approx ~15 new lines / ~1.5KB.

For example, my server does not include a /time.php endpont, so when I was bot scraped by a bot, of these lines were appended.

# ...
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.005"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.01"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.025"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.05"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.1"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.25"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="0.5"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="1"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="2.5"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="5"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="10"} 1
public_api_http_requests_duration_seconds_bucket{endpoint="/time.php",method="GET",status="404",le="+Inf"} 1
public_api_http_requests_duration_seconds_sum{endpoint="/time.php",method="GET",status="404"} 0.000011828
public_api_http_requests_duration_seconds_count{endpoint="/time.php",method="GET",status="404"} 1
# ...
public_api_http_requests_total{endpoint="/time.php",method="GET",status="404"} 1

Then, repeat for 100+ other 404'ing endpoints, and you end up with a really large metrics response, resulting in hundreds of GB of egress on your /metrics endpoint (which has been kindly refunded by the Fly.io billing team).

Screenshot from 2024-01-26 15-03-36

Furthermore, a dedicated attacker could certainly exhaust significantly more resources simply by 404ing on more URLs

To resolve this issue, 404 responses from endpoints that fail to match should not be included in metrics. I believe this PR should resolve the issue?

Also recommend submitting an advisory to RustSec for this version of the package.

oscarhermoso avatar Jan 26 '24 07:01 oscarhermoso

Just found https://github.com/nlopes/actix-web-prom/pull/71 and exclude_status(actix_http::StatusCode::NOT_FOUND)

I have applied this temporary work around to my repo, but is not in the docs or readme

oscarhermoso avatar Jan 26 '24 08:01 oscarhermoso