metaflow
metaflow copied to clipboard
Set memory limit to equal memory request?
Currently Metaflow doesn't set the memory limit for jobs on Kubernetes. This can be an issue if there are two memory-hungry jobs running on the same node. If both flows together use more than is available on the node, then the node crashes with an OOM error which is difficult to debug.
I suggest that the memory limit be set to equal the memory request on Kubernetes to avoid this situation.