etl
etl copied to clipboard
owid
A compute graph for loading and transforming OWID's data
## One-liner Improvements to the new data update workflow. ## Context & details At high level, the new workflow is: 0. Staging preparation - Create branch, PR - Data work...
Update Ember's global electricity data, and update all energy datasets. **NOTE: This data is embargoed until the 8th of May at midnight UK time.** Fixes: https://github.com/owid/owid-issues/issues/1401 ## Main changes Previously,...
[↑ Tracking issue](https://github.com/owid/etl/issues/2580) We are working on a new data workflow. A core part of it will be the **Indicator Upgrader**, which will help us migrate indicators from an old...
Currently, [our page on world region definitions](https://ourworldindata.org/world-region-map-definitions) uses very old datasets that were manually uploaded. They are not totally consistent with our regions dataset. For example, OWID regions include "French...
After #1539 you can now use the new tool `wizard` to generate templates for your ETL steps. Use this issue to report bugs that you may encounter. If the issue...
It should be possible to use Wizard to create steps on staging servers. I'd find it convenient when working exclusively on stagings.
Following [this discussion](https://owid.slack.com/archives/C46U9LXRR/p1714732013312239), I realised that, if you make changes to a snapshot (either the .py or the .dvc file) and re-run ETL, no changes are detected. I have also...
In this issue we can list all external uses of ETL datasets. This issue will be completed once those uses are added as steps in the new `external` channel (see...