metaflow
metaflow copied to clipboard
Netflix
:rocket: Build and manage real-life ML, AI, and data science projects with ease!
**Summary** After experimenting with some other frontends for Metadata service ingress, supporting basic auth as a client-side communication mechanism seems like a worthwhile exercise. The implementation is just the addition...
This fixes https://github.com/Netflix/metaflow/issues/1769. A quick addition that looks if the METAFLOW_KUBERNETES_CPU, METAFLOW_KUBERNETES_MEMORY, METAFLOW_KUBERNETES_DISK environment variable is set and if so parses the values. If a value is detected as being...
introduces `logs scrub run_id/step/task_id` command for scrubbing logs of tasks. by default it deletes both stdout/stderr for the latest attempt of a task. Final feature set: - `logs [pathspec]` works...
TODOs: - [x] How to associate step names with environment IDs? - [x] Where are dependency lock files located? Since we need to delete those as well..
This should only be merged after https://github.com/Netflix/metaflow/pull/1732 Notes: we need `typeguard` which is apparently not available for Python 3.5 and 3.6 https://pypi.org/project/typeguard/
addresses: https://github.com/Netflix/metaflow/issues/1771 issue: `python hello.py batch step --help` command was breaking due to wrong type for click.Choice arg. change: setting default arg for `ubf-context` from None to "none" in "batch...
Adds support for on demand Docker images. Open for discussion open todo's - [x] add fallback to `conda` environment when executing locally with `--environment docker` - [x] add support for...
more context - https://outerbounds-community.slack.com/archives/C02116BBNTU/p1714594027201059 accessible through slack.outerbounds.co