sentry-python
sentry-python copied to clipboard
getsentry
ajax request blocked with sentry enabled
How do you use Sentry?
Sentry Saas (sentry.io)
Version
sentry-sdk==1.7.2
Steps to Reproduce
The minimal reproduction is unfortunately still a bit bulky. For faster and easier reproduction, I created a repo for this: https://github.com/theendlessriver13/nginx-internal-sentry (but for the code also see below in <detail>)
the tl;dr is:
- 1 flask app (
app.py) - nginx as reverse proxy
- the app renders a
datatablestable which gets its data via anajaxrequest. This request needs authentication (auth_requestspecified for the route innginx.conf)
Now when loading the index / page, the ajax-POST request never reaches the app. I can see it in the logs, but only from nginx, (sends a 499). This is always followed by Ignoring EPIPE.
nginx | 2022/07/20 14:14:41 [notice] 1#1: start worker process 29
app | [2022-07-20 14:14:56 +0000] [7] [DEBUG] GET /
nginx | 172.21.0.1 - - [20/Jul/2022:14:14:56 +0000] "GET / HTTP/1.1" 200 1001 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" "-"
app | [2022-07-20 14:14:57 +0000] [7] [DEBUG] GET /is_authenticated
nginx | 172.21.0.1 - - [20/Jul/2022:14:14:58 +0000] "POST /ajax_route HTTP/1.1" 499 0 "http://0.0.0.0:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" "-"
app | [2022-07-20 14:14:58 +0000] [7] [DEBUG] Ignoring EPIPE
The interesting part is, that disabling sentry (setting the dsn to None) fixes the issue and the table renders properly. Removing the auth_request also fixes the problem (with sentry enabled).
It is some weird combination with the sentry-sdk and nginx using auth_request and the ajax request being sent.
for reproduction, be sure to always use port 8080, so going through the reverse proxy, port 5000 intentionally bypasses it (table also renders fine).
Sorry for the vague reproduction, tbh I am quite lost, whether this is an issue with sentry, nginx or datatables... I am happy to try out different things if you could point me into the right direction. Thank you for time.
to make the issue complete/persistent, I put my code also here (cloning the repo should be easier though to get it setup)
app.py
import sentry_sdk
from flask import Flask
from flask import Response
from sentry_sdk.integrations.flask import FlaskIntegration
sentry_sdk.init(
dsn=None,
integrations=[FlaskIntegration()],
traces_sample_rate=1.0,
)
app = Flask(__name__)
@app.route('/')
def index() -> Response:
return '''
<!DOCTYPE html>
<html lang="en">
<head>
<link
rel="stylesheet"
type="text/css"
href="https://cdn.datatables.net/v/dt/jq-3.6.0/dt-1.11.3/b-2.1.1/b-html5-2.1.1/sl-1.3.4/datatables.min.css"
/>
<script
type="text/javascript"
src="https://cdn.datatables.net/v/dt/jq-3.6.0/dt-1.11.3/b-2.1.1/b-html5-2.1.1/sl-1.3.4/datatables.min.js"
></script>
<title>Test</title>
</head>
<body>
<table id="table_id">
<thead>
<tr>
<th>firstname</th>
<th>lastname</th>
</tr>
</thead>
</table>
<script>
$(document).ready(function () {
let table = $("#table_id").DataTable({
ajax: { type: "POST", url: "/ajax_route", timeout: 1000, },
serverSide: true,
columns: [
{ data: "firstname", render: DataTable.render.text() },
{ data: "lastname", render: DataTable.render.text() },
],
});
});
</script>
</body>
</html>
'''
@app.route('/is_authenticated')
def auth() -> Response:
return 'OK'
@app.route('/ajax_route', methods=['GET', 'POST'])
def ajax() -> Response:
return {
'draw': 1,
'data': [
{'firstname': 'test', 'lastname': 'testing'},
{'firstname': 'test2', 'lastname': 'testing2'},
],
'recordsTotal': 2,
'recordsFiltered': 2,
}
if __name__ == '__main__':
app.run()
Dockerfile
FROM python:3.9-bullseye
WORKDIR /usr/src/app
COPY ./requirements.txt .
RUN python -m venv venv
RUN venv/bin/pip install --no-cache-dir -r requirements.txt
docker-compose.yml
version: "3"
services:
app:
container_name: app
build:
context: .
dockerfile: ./Dockerfile
image: app
user: nobody
ports:
- 5000:5000
volumes:
- ./app.py:/usr/src/app/app.py
command: [/usr/src/app/venv/bin/gunicorn, --bind, "0.0.0.0:5000", --log-level, debug, "app:app"]
nginx:
container_name: nginx
image: nginx:mainline
ports:
- 8080:8080
volumes:
- ./nginx.conf:/etc/nginx/conf.d/nginx.conf:ro
depends_on:
- app
nginx.conf
server {
listen 8080;
location / {
proxy_pass http://app:5000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_redirect off;
}
location /is_authenticated {
internal;
proxy_pass http://app:5000/is_authenticated;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_redirect off;
}
location /ajax_route {
auth_request /is_authenticated;
proxy_pass http://app:5000/ajax_route;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_redirect off;
}
}
requirements.txt
flask
gunicorn
sentry-sdk[flask]
Expected Result
I'd expect the ajax request to be send properly with the sentry-sdk enabled (reverse proxy setup, auth_request enabled)
Actual Result
The request is never processed by the flask app, therefore the table does not load
log-output from docker-compose:
nginx | 2022/07/20 14:14:41 [notice] 1#1: start worker process 29
app | [2022-07-20 14:14:56 +0000] [7] [DEBUG] GET /
nginx | 172.21.0.1 - - [20/Jul/2022:14:14:56 +0000] "GET / HTTP/1.1" 200 1001 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" "-"
app | [2022-07-20 14:14:57 +0000] [7] [DEBUG] GET /is_authenticated
nginx | 172.21.0.1 - - [20/Jul/2022:14:14:58 +0000] "POST /ajax_route HTTP/1.1" 499 0 "http://0.0.0.0:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0" "-"
app | [2022-07-20 14:14:58 +0000] [7] [DEBUG] Ignoring EPIPE
@theendlessriver13 hmm that's a very complex setup, but thanks for the detailed repro, appreciate it. I'll try to take a look when I have some time.
I had another go with this and tried both, the gunicorn and the werkzeug wsgi server, both with the same result, so we can exclude gunicorn, from the issue.
Another thing which is interesting however is, that this only happens when using type: 'POST', but not when using type: 'GET' in the ajax call - maybe I'm doing something wrong or I'm missing some headers - no idea...
just wanted to add this for completeness when I come back some time later
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀
![getsantry[bot] avatar](https://avatars.githubusercontent.com/in/66573?v=4 "Published by a pub.dev verified publisher"){kind=small}