Concept: link back from sources text to scripts that generated data

[larsyencken]

Core problem

We put a lot of effort into doing reproducible data management, so that our work is transparent to the public. We have two reasons for doing this:

1. Increase public trust in our work
2. Decrease the chances of a major error remaining in our work, and that error propagating into poor decisions and policies by decision makers

However, we do not explicitly link our final data visualisation with our data transformation steps, meaning it's nearly impossible for someone to find the relevant technical work or to review it.

Proposed solution

We could (somewhere) render a technical summary of how the data was produced, including links to the various Github scripts, and an indication of how to file an issue if they believe a mistake has been made.

This summary could be linked to in the Sources text of a chart/explorer, or whatever the evolution of that becomes in our data catalog idea.

Context

The making of datasets

A lot of steps can be involved in making a a dataset. In general, it looks like a graph like below:

When looking backwards from the perspective of the dataset you created, it looks like a tree. So some kind of tree-like hierarchy (or navigation) of these steps is necessary to accurately depict them.

Technical strategy

An Engineering theme for 2022 has been to try to encourage more community around our open source work. Bringing relevant people from the site back to our open source projects would meet this goal by encouraging the slow formation of a community of people interested in our data and in data management.

[larsyencken]

Shared this one already in the #proj-browsable-catalog slack channel

[github-actions[bot]]

This issue has had no activity within 10 months. It is considered stale and will be closed in 7 days unless it is worked on or tagged as pinned.

[larsyencken]

This has been folded into data pages thinking. It will live or die there.