etl
etl copied to clipboard
:honeybee: try replacing poetry with uv package manager
Background
uv
is a very fast Python package manager. It does some nice things:
- It replaces
pyenv
by using its own redistributable Python veresions - It picks package versions for all platforms at once (MacOS, Windows, Linux)
- It's just very very fast 🚀
- (perhaps) It doesn't randomly fail in CI pipelines? 😬
Proposal
We'd like to switch to uv
, since moving to ruff
has already been so good, and we think it could fix occasional CI issues.
Check out the slack discussion.
Todo
- [ ] Update the ETL and sub-packages to use
uv
- [ ] Update their Github CI pipelines
- [ ] Update the
ops
repo templates to installuv
- [ ] Tell all data folk to (
brew install uv
) - [ ] Batch install
uv
for all staging servers - [ ] Manually update all Buildkite pipelines