etl icon indicating copy to clipboard operation
etl copied to clipboard

Skip chart revision for charts with identical data

Open larsyencken opened this issue 11 months ago • 2 comments

What

Improve the chart revision flow by skipping charts that have no differences.

Why? Why now?

Currently, charts are queued up for manual review even when there are no changes, e.g. because other variables in a dataset changed.

We should make this flow more efficient by only requiring QA when there is something to QA.

Technical notes

  • Aren't there already options for auto-applying some chart revisions in staging sync?
    • Yes, in staging sync this is to avoid a double approval workflow
  • The chart revision code is the last piece for Daniel to migrate TypeORM-to-Knex database code; should he do it, or should we migrate it to Streamlit? Unclear
  • Decision: we should do this not in the chart approval code but in the code that decides whether to queue a chart for approval

larsyencken avatar Mar 08 '24 16:03 larsyencken

~~⏸️ We decided to park this until the next data engineering cycle, when we might migrate chart revision to Streamlit. We will also push for #data-viz time to get Grapher redistributable, so that owid-grapher-py can be our interface there.~~

larsyencken avatar Mar 11 '24 13:03 larsyencken

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar May 13 '24 20:05 stale[bot]

This is simple to do and still useful

danyx23 avatar May 21 '24 07:05 danyx23

But maybe we should move it over to the grapher repo as this is where this logic lies?

danyx23 avatar May 21 '24 07:05 danyx23

We plan to deprecate chart revisions in favour of chart-diff (which already has this feature). What is "worth QA-ing" in general is a bigger question we shall open soon after.

Marigold avatar May 21 '24 08:05 Marigold

Now it's a problem for chart-diff, we can track it there.

larsyencken avatar Jul 11 '24 09:07 larsyencken