datafusion-ballista
datafusion-ballista copied to clipboard
Ballista cluster sharing by multiple users (with fairness of resource allocation)
Is your feature request related to a problem or challenge? Please describe what you are trying to do. Let's say we've an organization that deploys ballista cluster to which multiple users connect to. It'd be great if executor slots can be allocated to the users fairly. If a query has consumed all slots allotted to a user, it can't progress until more slots are generated (say by passage of time).
Describe the solution you'd like Each query execution is associated with some identity, identities are rate limited for the slots and tasks for the identities are queued and consumed at allowed rate (or concurrency)
Describe alternatives you've considered
Additional context Add any other context or screenshots about the feature request here.