allure-docker-service
allure-docker-service copied to clipboard
K8s Deployment get restarted because readiness probe timeout under heavy load.
Good day,
I'm experiencing an issue on allure-docker-service running on K8s when our CI/CD pipelines try to upload test results for multiple projects in parallel. Apparently when under heavy load, serving multiple requests, readiness probes goes in timeout and so the POD is restarted.
I suspect that the server itself is able to handle only one request concurrently (or maybe a limited amount of requests).
There is any way to increase the number of requests concurrently handled by the service?
Or maybe using some http server like gunicorn in the docker image instead of launching the flask application directly could help in this sense.
If it makes sense, I can try to push a PR to implement that.
Thank you!
When I log in in my POD during this heavy load periods, I can see that the server itself is generating a lot of requests to generate the emailable report:
As a temporary fix, is it possible to avoid the generation of the email report using some env variable? I see in the script that it retries up to 7 times, making the server overloaded.
Thank you.
Please, check the documentation here:
Use env var CHECK_RESULTS_EVERY_SECONDS =NONE
https://github.com/fescobar/allure-docker-service?tab=readme-ov-file#MULTIPLE-PROJECTS---REMOTE-REPORTS