etl icon indicating copy to clipboard operation
etl copied to clipboard

:honeybee: try replacing poetry with uv package manager

Open Marigold opened this issue 5 months ago • 1 comments

Background

uv is a very fast Python package manager. It does some nice things:

  • It replaces pyenv by using its own redistributable Python veresions
  • It picks package versions for all platforms at once (MacOS, Windows, Linux)
  • It's just very very fast 🚀
  • (perhaps) It doesn't randomly fail in CI pipelines? 😬

Proposal

We'd like to switch to uv, since moving to ruff has already been so good, and we think it could fix occasional CI issues.

Check out the slack discussion.

Todo

  • [ ] Update the ETL and sub-packages to use uv
  • [ ] Update their Github CI pipelines
  • [ ] Update the ops repo templates to install uv
  • [ ] Tell all data folk to (brew install uv)
  • [ ] Batch install uv for all staging servers
  • [ ] Manually update all Buildkite pipelines

Marigold avatar Sep 09 '24 11:09 Marigold