AutSPACEs icon indicating copy to clipboard operation
AutSPACEs copied to clipboard

[FEATURE]: Create management command to export research data

Open llewelld opened this issue 11 months ago • 1 comments

Feature for…

Backend

What is the problem you want to solve?

One of the purposes of the AutSPACEs platform is to support the sharing of data for research. However, not all of the data on the platform is available for research (e.g. the author may not have consented to its use), the data is split across platforms (AutSPACEs database and OpenHumans) and it's not in a suitable format for a researcher to use.

The plan is therefore that the AutSPACEs team would do regular dumps of the data to create a "research grade data set" that could be used by researchers.

A method is therefore needed to extract a snapshot of the relevant data in an appropriate format usable for research.

So effectively that would be AutSPACEs running a django management command to grab the research-data from OH and then saving it locally for packaging as a data release.

How do you want to solve the problem?

  1. We propose solving the problem by making a django management command available to grab the research-data from OH, plus any additional data neeed from the AutSPACEs database, to be saved locally for packaging as a data release.
  2. An alternative would be to provide similar functionality through the Web UI, rather than as a django management command.
  3. Using a django management command is the prefered way for the project team, and helps maintain a clear separation between the user and management functionalities of the platform.

Other comments

Documentation for creating python management commands:

https://docs.djangoproject.com/en/4.2/howto/custom-management-commands/

This work isn't a current priority, but would be a neat way to get first-hand experience of django management commands.

llewelld avatar Aug 04 '23 13:08 llewelld