windmill
windmill copied to clipboard
windmill-labs
Unable to run docker container as unprivelleged (with "user:" option)
Describe the bug
When attempting to run windmill.dev with user: in the docker compose call, the following backtrace is produced:
Attaching to windmill
windmill | 2024-08-03T20:40:16.646786Z INFO src/main.rs:125: jemalloc enabled
windmill | 2024-08-03T20:40:16.646804Z INFO src/main.rs:196: Binary is in 'standalone' mode
windmill | 2024-08-03T20:40:16.646810Z INFO src/main.rs:267: Connecting to database...
windmill | 2024-08-03T20:40:16.663825Z INFO src/main.rs:269: Database connected
windmill | 2024-08-03T20:40:16.664596Z INFO src/main.rs:273: PostgreSQL version: PostgreSQL 16.3 (Debian 16.3-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit (windmill require PG >= 14)
windmill | 2024-08-03T20:40:16.666040Z INFO windmill-api/src/db.rs:77: Acquiring global PG lock for potential migration with pid: Some(190383)
windmill | 2024-08-03T20:40:16.666572Z INFO windmill-api/src/db.rs:97: Acquired global PG lock
windmill | 2024-08-03T20:40:16.667785Z INFO windmill-api/src/db.rs:112: Releasing PG lock
windmill | 2024-08-03T20:40:16.668029Z INFO windmill-api/src/db.rs:120: Released PG lock
windmill | 2024-08-03T20:40:16.668052Z INFO src/main.rs:311:
windmill | ##############################
windmill | Windmill Community Edition v1.371.4-1-g1a4732505
windmill | ##############################
windmill | 2024-08-03T20:40:16.668070Z INFO src/main.rs:711: config: MODE: standalone, BASE_URL: example.com, GO_PATH: /usr/local/go/bin/go, PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin, HOME: /
windmill | 2024-08-03T20:40:16.670576Z INFO windmill-common/src/worker.rs:497: Loading config from WORKER_GROUP: default
windmill | 2024-08-03T20:40:16.671093Z INFO src/monitor.rs:950: Reloading worker config...
windmill | 2024-08-03T20:40:16.672391Z INFO windmill-common/src/worker.rs:166: Loaded setting custom_tags, common: ["chromium"], per-workspace: {}
windmill | 2024-08-03T20:40:16.673079Z WARN windmill-api/src/oauth2_ee.rs:180: oauth.json not found, no OAuth clients loaded
windmill | 2024-08-03T20:40:16.674247Z WARN windmill-common/src/server.rs:77: SMTP not configured
windmill | 2024-08-03T20:40:16.674253Z INFO src/monitor.rs:907: Reloading server config...
windmill | 2024-08-03T20:40:16.674435Z INFO src/monitor.rs:731: Loaded setting retention_period_secs from db config: Number(2592000)
windmill | 2024-08-03T20:40:16.674793Z INFO src/monitor.rs:705: Loaded saml_metadata setting to None
windmill | 2024-08-03T20:40:16.674954Z INFO src/monitor.rs:705: Loaded scim_token setting to None
windmill | 2024-08-03T20:40:16.675086Z INFO src/monitor.rs:705: Loaded pip_extra_index_url setting to None
windmill | 2024-08-03T20:40:16.675249Z INFO src/monitor.rs:705: Loaded pip_index_url setting to None
windmill | 2024-08-03T20:40:16.675443Z INFO src/monitor.rs:705: Loaded npm_config_registry setting to None
windmill | 2024-08-03T20:40:16.675636Z INFO src/monitor.rs:705: Loaded bunfig_install_scopes setting to None
windmill | 2024-08-03T20:40:16.704028Z INFO windmill-api/src/embeddings.rs:222: Loading embedding model...
windmill | 2024-08-03T20:40:16.704263Z INFO src/main.rs:704: Successfully connected to pg listen
windmill | 2024-08-03T20:40:16.734714Z INFO windmill-api/src/lib.rs:365: server started on port=8000 and addr=0.0.0.0 instance=Ocbuu
windmill | 2024-08-03T20:40:17.032245Z ERROR windmill-api/src/embeddings.rs:173: Failed to get config.json from hugging face: I/O error Permission denied (os error 13)
windmill | 2024-08-03T20:40:17.032686Z ERROR windmill-api/src/embeddings.rs:613: Failed to initialize model instance: could not get config.json
windmill | thread 'main' panicked at src/main.rs:790:14:
windmill | could not create initial worker dir: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
windmill | stack backtrace:
windmill | 0: 0x56309c11ca55 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1e1a1972118942ad
windmill | 1: 0x56309c14d2ab - core::fmt::write::hc090a2ffd6b28c4a
windmill | 2: 0x56309c117f1f - std::io::Write::write_fmt::h8898bac6ff039a23
windmill | 3: 0x56309c11c82e - std::sys_common::backtrace::print::ha96650907276675e
windmill | 4: 0x56309c11dca9 - std::panicking::default_hook::{{closure}}::h215c2a0a8346e0e0
windmill | 5: 0x56309c11d9ed - std::panicking::default_hook::h207342be97478370
windmill | 6: 0x56309c11e143 - std::panicking::rust_panic_with_hook::hac8bdceee1e4fe2c
windmill | 7: 0x56309c11e024 - std::panicking::begin_panic_handler::{{closure}}::h00d785e82757ce3c
windmill | 8: 0x56309c11cf19 - std::sys_common::backtrace::__rust_end_short_backtrace::h1628d957bcd06996
windmill | 9: 0x56309c11dd57 - rust_begin_unwind
windmill | 10: 0x5630955e9fd3 - core::panicking::panic_fmt::hdc63834ffaaefae5
windmill | 11: 0x5630955ea486 - core::result::unwrap_failed::h82b551e0ff2b2176
windmill | 12: 0x5630961c825b - windmill::windmill_main::{{closure}}::{{closure}}::h12b89e7fff7a7230
windmill | 13: 0x563095ea441b - <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::he50395f72f8e67a7
windmill | 14: 0x56309565a657 - windmill::windmill_main::{{closure}}::hdb2e0a23aa4b29e5
windmill | 15: 0x56309562f0e9 - tokio::runtime::park::CachedParkThread::block_on::hfc7208a502545d5e
windmill | 16: 0x563095e3a94f - tokio::runtime::context::runtime::enter_runtime::hfe3d9095ee3dd6d6
windmill | 17: 0x56309597c99c - tokio::runtime::runtime::Runtime::block_on::h394cec4798ae5607
windmill | 18: 0x563095ee746d - windmill::main::hd78c1289a1d9d719
windmill | 19: 0x563095c4c163 - std::sys_common::backtrace::__rust_begin_short_backtrace::h04bff02b80bf23fa
windmill | 20: 0x563095c248fd - std::rt::lang_start::{{closure}}::hb1f6a40ebc40ef91
windmill | 21: 0x56309c10e760 - std::rt::lang_start_internal::h3ed4fe7b2f419135
windmill | 22: 0x563095ee7be5 - main
windmill | 23: 0x7f4e3f27824a - <unknown>
windmill | 24: 0x7f4e3f278305 - __libc_start_main
windmill | 25: 0x5630955ead91 - _start
windmill | 26: 0x0 - <unknown>
windmill exited with code 101
To reproduce
include user: 1000:1000 (substitute for relavant UID/GID) in docker-compose.yml
Expected behavior
The ability to run windmill as an unprivileged container
Screenshots
No response
Browser information
No response
Application version
No response
Additional Context
No response
Just ran into this as well, I think the worker node is trying to create /tmp/windmill/cache_nomount, but it can't do that without root perms since /tmp/windmill is most likely owned by root. Since /tmp/windmill/cache and /tmp/windmill/logs come mounted from the host, they're likely owned by the non-root user and therefore have no issue read/writing to them
Hitting this problem too.
As a note I'm checking on existing pods, the folders are there with correct permissions:
I have no name!@windmill-workers-6b96cfdb69-bm5sw:/usr/src/app$ ls -l /tmp/windmill
total 0
drwxr-xr-x 11 1000 1000 115 Aug 27 06:48 cache
drwxr-xr-x 4 1000 1000 31 Aug 27 06:48 cache_nomount
drwxr-xr-x 2 1000 1000 6 Aug 27 06:48 logs
drwxr-xr-x 3 1000 1000 50 Aug 27 06:48 wk-bm5sw-nrvwz
I have no name!@windmill-workers-6b96cfdb69-bm5sw:/usr/src/app$ ls -ld /tmp/windmill
drwxr-xr-x 6 1000 1000 95 Aug 27 06:48 /tmp/windmill
This is true at least until version 1.380.0 (helm chart 2.0.253)
It must have changed how the main /tmp/windmill folder is created, possibly it's now done by root and then creating subfolders is denied for user 1000.
This is solved on latest and on next release 1.386.0. /tmp/windmill was created in Dockerfile and not chmod correctly
