windmill-labs
feature: Improve packaging/distribution of components?
I am trying to write a Windmill module for NixOs. It would be really nicer if there were separate independent packages for the different components of this app. I have a been able to build the windmill cli imperatively by following the Dockerfile provided but it still expects something on the database end (Error: Migrating database: while executing migrations: error returned from database: role "admin" does not exist).
It would be very easy to package if the frontend and backend (and if it needs anything else) can be run with cli flags independently.
Please let me know if you need more details.
I will follow up on that, it's easier than one would think. That error is strange as it should auto setup the admin role for you but I will take a look at it if there is an easy way to reproduce this.
is there anyway that we can avoid creating an admin role in Postgres?
I use a shared Postgres instance between multiple services so granting windmill admin roles is not so safe
@vanhtuan0409 Windmill needs to apply migrations that create the tables and we use the admin role mostly to bypass our own Row Level Services for granular user permissions. Would you prefer applying the migrations by hand ? The admin is only admin wrt to Windmill and not wrt to the database. I could theoretically make that name configurable.
@dit7ya Due to a mistake that I did when I wrote the first migration, you would need to pre-create the admin role in the postgres database that you have a DATABASE_URL of. I know it's pretty annoying but it's not so easy to rewrite the migration as they are compared by checksum. I will need to mitigate this one way or another but is there a way the NixOs setup could use psql to add the role for now ?
About the frontend and backend, they do not need to be packaged separately because the backend serves the frontend. If you want to build from source, you need to do npm run build before you do cargo build as the cargo step will embed the frontend as a static assets served by it.
Pretty much the instructions are the same than the local setup: https://github.com/windmill-labs/windmill#backend--frontend except instead of doing npm run dev it's npm run build, and cargo build instead of cargo run.
@dit7ya Something I just realized, if you are only trying to build the binary: SQLX_OFFLINE=true will make it that you do not need to have a database up to create the binary. It's one ENV flag that is easy to skip in the Dockerfile. If you have any other issues, we can also chat on Discord https://discord.com/invite/V7PM2YHsPB, having a Nix package would be very cool!
@rubenfiszel that you for your reply. Generally speaking, I expected that the user configured in DATABASE_URL only has privilege over the windmill database, without the ability to create new roles, set password, etc... For the admin and user roles, I would prefer to create them myself. And it would be great if those roles could be a little more specific like windmill_admin and windmill_user. The rest of the migration can still be run automatically. Currently, this is the only issue that keep me away from running windmill
Anw, great project. Kudo to you guys
@vanhtuan0409 I created #403 #402 to track this. I should have thought of this earlier. The way migrations are done it is not so easy to alter past ones but I will investigate.
@vanhtuan0409 @dit7ya I have applied the necessary changes in #404 and added instructions to compile from source and to run without using a superuser here: https://docs.windmill.dev/docs/how-tos/self_host
I have been able to create a WIP derivation for the CLI (with the latest commit) but it seems to fail with the following error when I try to run it with a fresh postgres db initialized via bit.io.
❯ DATABASE_URL=postgresql:/<REDACTED>@db.bit.io/dit7ya/third ./result/bin/windmill
Error: Migrating database: while executing migrations: error returned from database: syntax error at or near "-"
Caused by:
0: while executing migrations: error returned from database: syntax error at or near "-"
1: error returned from database: syntax error at or near "-"
2: syntax error at or near "-"
@dit7ya Is it possible that bit.io is not running Postgres 14 or have some limitations ? The fact that it returns a syntax error is really weird. CI runs those migrations to do the backend integration tests as well as the playwright tests. I have also tested locally and there is no syntax error anywhere. Can you make it work with a standard PG 14 ?
Seems like it does not return any error with a local Postgres db and I have been able to get the following screen
Is there some way to get the first login without adding OAuth?
I want to test some basic functionalities before creating a draft PR to nixpkgs. I am especially not confident since right now I have the cargo tests disabled as those will need a patch for libssl.so (but not yet sure exactly where).
Thanks, worked!
Now I am getting these errors whenever I try to do anything
2022-08-14T19:35:13.317603Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table workspace"
2022-08-14T19:35:15.415680Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table workspace"
2022-08-14T19:35:15.483533Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table workspace"
2022-08-14T19:35:15.598810Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table workspace"
2022-08-14T19:35:15.682374Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table workspace"
2022-08-14T19:35:17.629222Z ERROR windmill::error: error="Sql error: error returned from database: permission denied for table script"
2022-08-14T19:35:39.594472Z ERROR windmill::error: error="Internal: fetching if ting is premium: error returned from database: permission denied for table workspace"
2022-08-14T19:35:58.875403Z ERROR windmill::error: error="Internal: fetching if ting is premium: error returned from database: permission denied for table workspace"
I am guessing it has something to do with the postgres user?
Are you using a different postgres user for connecting to the db than the one you ran the migration with ?
I have not ran any migrations manually - just used a fresh db with the binary.
Yes but the binary runs the migrations for you with the DATABASE_URL you first provided if the migrations have not been applied yet. Is the user that you used in that URL an owner of that database ?
If the user of the DATABASE_URL is not a super user, you need to run the init-db script indicated by: https://docs.windmill.dev/docs/how-tos/self_host#run-windmill-without-using-a-postgres-superuser by a superuser at least once.
Sorry, I am too confused here since I am not familiar with postgres administration. I ran the script ( sudo -u postgres psql postgres://postgres:test@localhost/windmill4 -f init-db-as-superuser.sql) and this is what \du looks like now
List of roles
Role name | Attributes | Member of
----------------+------------------------------------------------------------+-----------
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
superuser | | {}
windmill_admin | Cannot login, Bypass RLS | {}
windmill_user | Cannot login | {}
I am guessing again something is wrong as I am getting the same errors.
However, I believe this is something on my end and I will be able to figure it out soon. Thanks for all your help.
It might not necessarily be on your end, but you have a superuser that is not a superuser in your \du. Are you by any chance connecting to postgres with superuser in your DATABASE_URL ?
Nvm I was able to reproduce this on a fresh install, let me investigate this, it's definitely not on your end
@dit7ya clean your db, take the latest commit, it should work now, it was a bad configuration stemming from #404 changes. Sorry about that.
Thanks. Not getting those errors anymore. But when I try to preview a script getting these errors
job 0182a075-6088-9ce1-4b8b-2670a98689c3 on worker dt-worker-H0rfs-mac4X
--- DENO CODE EXECUTION ---
IO error: No such file or directory (os error 2)
I don't see any other logs anywhere else.
nsjail is in PATH and getting the same error with DISABLE_NSJAIL=true DATABASE_URL=postgres://postgres:test@localhost/windmill ./result/bin/windmill as well. I think windmill is trying to call deno by some hardcoded path (nix does not follow FHS)? Can you point me to the code where that happens?
The path that it expects is /usr/bin/deno and /usr/local/bin/python3. I can make that env variable configurable in the worst case.
@dit7ya in latest commit: 92d1a3a, I removed the hardcoded paths when NSJAIL is disabled in favor of getting "python3" and "deno" from the path.
I am trying deno to work with the latest commit but it is still failing whereas it should not. (Python needs more patching since installing those pip packages won't work in NixOS).
Can you tell me how do I get more logs about why deno is failing (as in what was the full command that was sent and any other info). You don't have to commit anything to this repo yet. Just tell me what to add in backend/src/worker.rs and I will that as a patch in the build step.
If you disable NSJAIL, the full command that is run is here:
https://github.com/windmill-labs/windmill/blob/main/backend/src/worker.rs#L830
deno run --unstable -v8-flags=--max-heap-size=2048 main.ts
where the command is run from a folder specific to this worker and job inside the hardcoded tmp folder "/tmp/windmill":
https://github.com/windmill-labs/windmill/blob/main/backend/src/worker.rs#L49
I think I figured out the issue. You are clearning the env variables at https://github.com/windmill-labs/windmill/blob/1275f5f7fb65e32a17d7d397d43d0b49ecd5cd0e/backend/src/worker.rs#L832 which means PATH gets erased as well.
Maybe a better idea is to use something like which deno?
Good catch, in latest commit I have made the paths as env variable configurable at:
DENO_PATH
PYTHON_PATH
NSJAIL_PATH
Would that work for packaging ? Using which at runtime seems a bit odd to me but that might work too.