cubed
cubed copied to clipboard
Prefect serverless work pool executor
One of Matt Rocklin's blog posts mentions Prefect's "serverless work pool"
https://docs.prefect.io/v3/deploy/infrastructure-examples/serverless
My (hazy) understanding of what this is is a way to deploy a cluster in the cloud on-demand and submit work to it without having a client process controlling it at all times. This seems like it could be useful as a way to provision a cluster on which to run an in-memory rechunk without having to worry about turning off the cluster again.