Gunicorn workers time out on event handler in prod mode
Describe the bug
When starting reflex in production mode below example freezes inside the pd.read_csv call until the guniorn workers time out and with below message:
[2024-07-01 18:09:21 +0200] [92618] [CRITICAL] WORKER TIMEOUT (pid:92671)
[2024-07-01 18:09:21 +0200] [92618] [ERROR] Worker (pid:92671) was sent SIGABRT!
To Reproduce Steps to reproduce the behavior:
- Code/Link to Repo:
import polars as pl
import reflex as rx
csv_text = """col1,col2
1,2""".encode(
"utf-8"
)
df = pl.read_csv(csv_text)
print("module df", df)
class BugState(rx.State):
def on_load(self):
print("Bug page on load")
df = pl.read_csv(csv_text)
print("loaded df")
print(df)
@rx.page(route="/test", on_load=BugState.on_load)
def bug() -> rx.Component:
return rx.text("Hello")
start reflex in production mode: reflex run --env prod
Expected behavior
A clear and concise description of what you expected to happen.
code in on_load runs to completion and workers don't time out
Screenshots If applicable, add screenshots to help explain your problem.
Specifics (please complete the following information):
- Python Version: 3.11.8
- Reflex Version: 0.5.5
- OS: MacOS 14.5 (23F79) (also unix when deployed on AWS)
- Browser (Optional):
Additional context Add any other context about the problem here.
I just came across this so this might be a multiprocessing issue in the way that the gunicorn workers are started: https://docs.pola.rs/user-guide/misc/multiprocessing/
I've been playing around with this more. It looks like this issue is caused if polars is touched on module level.
That is, if the module level pl.read_csv is removed the code will work.
it seems a polar issue, not much we can do on our side