metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Set memory limit to equal memory request?

Open tylerpotts opened this issue 1 year ago • 0 comments

Currently Metaflow doesn't set the memory limit for jobs on Kubernetes. This can be an issue if there are two memory-hungry jobs running on the same node. If both flows together use more than is available on the node, then the node crashes with an OOM error which is difficult to debug.

I suggest that the memory limit be set to equal the memory request on Kubernetes to avoid this situation.

tylerpotts avatar Sep 19 '24 19:09 tylerpotts