mu icon indicating copy to clipboard operation
mu copied to clipboard

Add early checks for bad configurations in lambdize pipelines

Open shelbyt opened this issue 8 years ago • 3 comments

Example:

If a __server.py binary runs with a bucket that does not exist (-b option). Errors are thrown _after* all the lambdas are configured.

SERVER HANDLING (1) FAIL(retrieving norun:sintel-1k-png16/00000007.png->/tmp/lambda_Sc4R9q/00000007.png from s3: Traceback (most recent call last):

Might be better to catch bad configurations early on before we start invoking any lambdas.

shelbyt avatar Oct 11 '16 18:10 shelbyt

Slightly disorganized thoughts:

One minor thing to worry about: implementing this check by having the server talk to S3 means that the server needs to be able to access the target S3 bucket (to verify its existence). In contrast, right now the server only needs permission to launch lambdas, and then the lambda role associated with the function being launched needs access to S3.

Implementing this check with a "trial balloon" worker is a possibility, but that's a big change given the way mu launches lambdas.

There are probably other ways to handle this, too.

kwantam avatar Oct 11 '16 19:10 kwantam

One more thought: this doesn't need to be the ~~worker's~~coordinator's responsibility. Before launching a job, a separate "checking" stage can catch errors like this. This is probably the best approach because it cleanly separates two tasks that do not need to be connected.

kwantam avatar Oct 12 '16 17:10 kwantam

(edited above: s/worker's/coordinator's/)

kwantam avatar Oct 12 '16 18:10 kwantam