metaflow
metaflow copied to clipboard
Netflix
:rocket: Build and manage real-life ML, AI, and data science projects with ease!
while there is https://github.com/Netflix/metaflow/blob/master/CONTRIBUTING.md I was looking for something like a dev env setup to be to add a feature and test it etc. For example, see the numpy one...
Documentation is missing information for local K8S deployment https://docs.metaflow.org/getting-started/infrastructure Point 5 links to AWS-K8S We're currently evaluating Metaflow, but would like to understand on premise requirements (or possibilities). In our...
After checked out the code, I saw that the value `OTEL_ENDPOINT` is only being set to the environment variable for Kubernetes, and cannot be found any where else within the...
When I run a flow from AWS Step Functions And a step throws an exception And the exception doesn't go away after retrying Then run.finished isn't True I'm triggering flows...
changes the Task ID generation for tasks in foreach branches to be deterministic. This is so that the following join steps do not require an enormous list of ID's to...
The task_id's of the preceding foreach steps for a join appear multiple times in the Argo Workflow init containers `ARGO_TEMPLATE`, which bloats up the size significantly. With wide foreaches the...
When deploying into production we generate AWS resources using: ``` python .py --with retry step-functions create ``` The most basic workflow: ``` from metaflow import FlowSpec, step class TestFlow(FlowSpec): @step...
Use case: After a metaflow job ends, would like the flow of control to go back to the microservice that triggered the metaflow job For a general case, lets say,...
**Use Case**: Be notified on Flow failure, say a daily Flow, or on success for notification. A `@notify()` decorator that has Metaflow send a notification upon success or failure, per...
There are some problematic names for steps due to an open bug with Argo Workflows: https://github.com/argoproj/argo-workflows/issues/12037 This affects even the provided tutorials, where a `join` step causes the flow to...