owid-grapher icon indicating copy to clipboard operation
owid-grapher copied to clipboard

Project: use the data api for baking and the grapher admin

Open danyx23 opened this issue 2 years ago • 0 comments

What

We now have a data api that is serving the data of our catalog that is built from the etl. This data api can serve variable data and metadata in json format similar to what we bake as static files that grapher uses for displaying charts. This project is about making it possible that baking and the grapher admin get what they need from the data api.

Why

Currently both the baking process and the grapher admin fetch the variable data and metadata json directly from Grapher's mysql database (via the admin backend for the admin). Because of this we still need to put all data values into mysql. If we could fetch the data (and maybe metadata) for baking and the grapher admin from the data api then we could stop putting all data values into mysql and make the database a lot leaner.

Why now

The data api is now in a state where this is feasible.

Steps

  • [x] Change admin backend to fetch data from data api instead of from the mysql database via the admin server
  • [x] Resolve any mismatches that still exist between the baked files and the api
  • [x] Use the data api when baking files

danyx23 avatar Aug 01 '22 12:08 danyx23