metaflow
metaflow copied to clipboard
Netflix
:rocket: Build and manage real-life ML, AI, and data science projects with ease!
An implementation to allow SFN Executions to be triggered from the finishing state of another SFN execution. I have been using this to trigger multiple consumer Flows when a specific...
I was wondering what is the significance of walking the current directory. https://github.com/Netflix/metaflow/blob/5186c6c5bba36d9e77077413ee2495dc79da3dca/metaflow/package.py#L81-L92 I was trying to run a flow from my home directory and a simple hello world flow...
When Python does not guarantee the order of a collection (such as a set), using such an object as a target for a `foreach` fan-out can cause incorrect execution with...
When an artifact fails to serialize, we should show a clear error message highlighting the specific artifact that caused trouble. Context: https://gitter.im/metaflow_org/community?at=5e29e840348292727916743a
This PR might be a bit controversial as it has quite a lot of plumbing to make PyTorch Lightning multiprocessing act nicely with Metaflow. This PR adds support for multinode...
Currently, you can do something like this: `Task(Flow/RunID/StepName)` and this will not result in an error but then the resulting `Task` object behaves in a bizarre manner where things like...
I do see issues and pull requests like this: - https://github.com/Netflix/metaflow/issues/50 - https://github.com/Netflix/metaflow/pull/644 - https://github.com/Netflix/metaflow/pull/488 Does this mean metaflow will run fully on our on-prem Kubernetes cluster or is there...
(previously #744) There seems to be several places where we convert arguments to strings, I'm not 100% sure how they are all used. Need this for K8S as some parameters...
Right now if one attempted to pass environment variables to the `@environment` decorator using the following type of syntax: ``` python workflow/myflow.py --with environment:vars=FOO:$(FOO),BAR:$(BAR) ``` vars would incorrectly register as...
