windmill
windmill copied to clipboard
windmill-labs
bug: backend crash when we hover the Edit button on the frontend UI
Describe the bug
This is a new instance I installed couple days ago and the only activities where sync'ing flows from another instance.
Whenever I hover the Edit button in a flow this happens:
Then I have to restart the container but somehow since this issue appeared the container doesn't start properly here is the log (notice the logs from src/monitor.rs:298 that are not printed in another healthy instance I have):
2024-06-24T16:45:27.237122Z INFO src/main.rs:125: jemalloc enabled
2024-06-24T16:45:27.237147Z INFO src/main.rs:157: Binary is in 'server' mode
2024-06-24T16:45:27.237172Z INFO src/main.rs:250: Connecting to database...
2024-06-24T16:45:27.264046Z INFO src/main.rs:252: Database connected
2024-06-24T16:45:27.268736Z INFO src/main.rs:256: PostgreSQL version: PostgreSQL 16.3 (Debian 16.3-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit (windmill require PG >= 14)
2024-06-24T16:45:27.269352Z INFO windmill-api/src/db.rs:78: Acquiring global PG lock for potential migration with pid: Some(1872)
2024-06-24T16:45:27.269833Z INFO windmill-api/src/db.rs:98: Acquired global PG lock
2024-06-24T16:45:27.273272Z INFO windmill-api/src/db.rs:113: Releasing PG lock
2024-06-24T16:45:27.273629Z INFO windmill-api/src/db.rs:121: Released PG lock
2024-06-24T16:45:27.273664Z INFO src/main.rs:294:
##############################
Windmill Community Edition v1.352.0-6-g189a2815a
##############################
2024-06-24T16:45:27.273691Z INFO src/main.rs:651: config: MODE: server, GO_PATH: /usr/local/go/bin/go, PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin, HOME: /root
2024-06-24T16:45:27.278900Z INFO windmill-common/src/worker.rs:99: Loaded setting custom_tags, common: ["chromium"], per-workspace: {}
2024-06-24T16:45:27.279352Z WARN windmill-api/src/oauth2_ee.rs:180: oauth.json not found, no OAuth clients loaded
2024-06-24T16:45:27.280508Z INFO src/monitor.rs:963: Reloading server config...
2024-06-24T16:45:27.280688Z INFO src/monitor.rs:697: Loaded setting retention_period_secs from db config: Number(2592000)
2024-06-24T16:45:27.281041Z INFO src/monitor.rs:671: Loaded saml_metadata setting to None
2024-06-24T16:45:27.281255Z INFO src/monitor.rs:671: Loaded scim_token setting to None
2024-06-24T16:45:27.281875Z INFO src/monitor.rs:298: 2.1177520751953125 mb allocated/11.92578125 mb resident
2024-06-24T16:45:27.321315Z INFO windmill-api/src/embeddings.rs:222: Loading embedding model...
2024-06-24T16:45:27.321743Z INFO src/main.rs:644: Successfully connected to pg listen
2024-06-24T16:45:27.321775Z INFO windmill-api/src/embeddings.rs:196: Found embedding model in cache
2024-06-24T16:45:27.384087Z INFO windmill-api/src/lib.rs:339: server started on port=8000 and addr=0.0.0.0 instance=IXGH7
2024-06-24T16:45:27.492744Z INFO windmill-api/src/embeddings.rs:238: Loaded embedding model
2024-06-24T16:45:27.492769Z INFO windmill-api/src/embeddings.rs:625: Creating embeddings DB...
2024-06-24T16:45:29.013147Z INFO windmill-api/src/embeddings.rs:632: Created embeddings DB
2024-06-24T16:45:57.284491Z INFO src/monitor.rs:298: 161.61193084716797 mb allocated/272.140625 mb resident
2024-06-24T16:46:27.286701Z INFO src/monitor.rs:298: 161.98434448242188 mb allocated/272.359375 mb resident
2024-06-24T16:46:57.288199Z INFO src/monitor.rs:298: 162.03257751464844 mb allocated/272.3359375 mb resident
2024-06-24T16:47:27.290323Z INFO src/monitor.rs:298: 162.0587921142578 mb allocated/272.3359375 mb resident
To reproduce
- Go to '...'
- Click on '....'
- Scroll down to '....'
- See error
Expected behavior
No crashes
Screenshots
No response
Browser information
No response
Application version
CE v1.352.0-6-g189a2815a
Additional Context
No response
It's either a cache issue or a load balancer misconfiguration issue. Either your asset requests are not going to your servers or the cached frontend you have is not the same version that your server provide (in which case the request logs of your servers would show lots of 404)
In what case would the server change the asset version? I didn't even restart the container before this issue appeared.
On Mon, 24 Jun 2024 at 19:45, Ruben Fiszel @.***> wrote:
It's either a cache issue or a load balancer misconfiguration issue. Either your asset requests are not going to your servers or the cached frontend you have is not the same version that your server provide (in which case the request logs of your servers would show lots of 404)
— Reply to this email directly, view it on GitHub https://github.com/windmill-labs/windmill/issues/3963#issuecomment-2187093467, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGJQUDNCAXUMZATK335KX3ZJBLJZAVCNFSM6AAAAABJ2GKMFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGA4TGNBWG4 . You are receiving this because you authored the thread.Message ID: @.***>
I misread the issue, could you print the service logs of the server that crashed and the logs of the container that doesn't start properly
So I realized the backend didn't crash or at least it didn't produce any logs, is there another log besides the one I shared above?
I do not know your setup so yes possibly? Anyway, a request for an asset didn't reach your server when it should have which is why the frontend is displaying that error.
That's really strange with only this .css file that fails. Do you happen to know if there's any changes that I can try on Cloudflare? I've already added a Page rule to bypass the cache but it didn't help. In any case thank you for taking a look, I'll try to redeploy and see if the issue comes back
On Mon, Jun 24, 2024, 21:43 Ruben Fiszel @.***> wrote:
I do not know your setup so yes possibly? Anyway, a request for an asset didn't reach your server when it should have which is why the frontend is displaying that error.
— Reply to this email directly, view it on GitHub https://github.com/windmill-labs/windmill/issues/3963#issuecomment-2187281942, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGJQUAZYKAHVDNF2HAWZTLZJBZHPAVCNFSM6AAAAABJ2GKMFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGI4DCOJUGI . You are receiving this because you authored the thread.Message ID: @.***>