dagster
dagster copied to clipboard
dagster-io
Support Auth & RBAC in Dagit
This is decidedly low priority for now, but just filing an issue to track for the future.
Eventually, Dagit should support authn/authz in dagit, permit assigning roles with different privileges, etc.
Any suggestions on how to implement this today? I am looking for some something simple, just to allow/disallow access to dagster.
Hey @calleo - it depends on your deployment context, if you're able to put a reverse proxy w/ some kind of auth in front of the Dagit host/container that's a straightforward way to get this.
On GCP, there's https://cloud.google.com/iap which we've used successfully w/ G Suite logins, on AWS there's https://aws.amazon.com/blogs/aws/built-in-authentication-in-alb/ which should let you hook into a SAML provider.
@natekupp and what about solid-level permissions?
@gardner-dev solid-level permissions is not something we've considered yet—can you say more about your use case and what you're looking for?
I meant pipelines, sorry. So basically I think direction in which Airflow went is really nice. Dagit could became somehow self-service, when specific user have access only to limited numbers of pipelines based on permissions/roles
Don`t try it, but workaround maybe can be for now to make pipelines with some variable "key" in the config that is required to run it.
I think something like admin panel with username and password as login is needed to secure the system.
how can we run a multi-tenant version of dagster without auth and permissions ?
+1 on this. We have a shared deployment and it would be nice to know who is running what jobs, and also restrict sensitive pipelines (ie anything that interacts with end user-facing parts of our infrastructure, versus basic ETL tasks that are sandboxed)
+1 on this - it is part of our considerations as to whether or not to adopt the tech
As of Dagster 0.11.10, we currently have an option to support basic RBAC by spinning up a read-only Dagit to prevent users from running mutations. As for run attribution, we have an example in the docs to introduce this by subclassing the queued run coordinator.
+1 on this - For us, something as simple as the token Jupyter Lab uses, is sufficient. Basically, you define a token as an ENV variable. And when calling dagit or the API, you can provide a token as a query parameter, as an Authorization header or via cookies.
I think Authn is higher priority than Authz/RBAC. Basically, the dagit panel will be only used by an admin user. I don't need to control the user permissions, but a basic username/password authentication is necessary. Just like Airflow.
RBAC might push us to use Prefect instead of Dagster, even though I really like Dagster's engineering. Basic auth to protect the UI is a must (and easier if we don't have to run a separate proxy for that). Read/run/update versus readonly views would be nice as a separation for owners versus data consumers. And pipeline- if not solid-level permissions would be great, for example so data scientists can update input parameters and re-run certain portions of the DAG or certain pipelines, while software is responsible for maintaining/deploying the overall pipeline, scheduling, etc.
Thanks for the ping on this @markfickett.
Right now we are offering RBAC and authentication through our cloud product, which is open to early access design partners. Feel free to sign up for the waitlist here (https://dagster.io/cloud). We'd love to chat about working with you.
+1 planning on using dagster in my new team and this would be an absolute must in the near future. A workaround with a reverse proxy would be usable but far from ideal.
+1 we are looking forward to having a login screen and a crud of users and permissions to manage our team's access to the dagit UI. We find it very insecure to leave our pipelines exposed within our corporate network, where any employee can modify our schedules.
This feature or a simpler subset of it would be greatly appreciated here as well !
Would this be implemented as a starlette middleware writing extra headers ?
This mecanism is used by the DagsterTracedCounterMiddleware adding an x-dagster-call-counts headers in dagit/webserver.py already, perhaps this is something we can help with ?
RBAC or simple HTTPS Auth would be a nice future.
Without that, of course we can use some ngix reverse proxy, but it's far from ideal.
