cubed icon indicating copy to clipboard operation
cubed copied to clipboard

Prefect serverless work pool executor

Open TomNicholas opened this issue 9 months ago • 2 comments

One of Matt Rocklin's blog posts mentions Prefect's "serverless work pool"

https://docs.prefect.io/v3/deploy/infrastructure-examples/serverless

My (hazy) understanding of what this is is a way to deploy a cluster in the cloud on-demand and submit work to it without having a client process controlling it at all times. This seems like it could be useful as a way to provision a cluster on which to run an in-memory rechunk without having to worry about turning off the cluster again.

TomNicholas avatar Jan 06 '25 20:01 TomNicholas