physionet-build
physionet-build copied to clipboard
Allow published_project.gcp.sent_files to be toggled (True/False) via the admin console
For large files, we often have to manually transfer the data to GCP using the approach outlined below. Either we:
- improve the process for transferring files to the cloud, so that it doesn't fail for large datasets or
- add a form field to the published-projects page in the console that allows the
published_project.gcp.sent_files
flag to be toggled.
Add data to Google Cloud bucket
First go to the project management page (e.g. https://physionet.org/console/published-projects//kinecal/1.0.3/) and click “Create bucket” to create a bucket on Google Cloud.
Now the bucket has been created, we can upload the files directly from the PhysioNet server using gsutil:
gsutil -m cp -r ./* gs://[kinecal-1.0.3.physionet.org](http://kinecal-1.0.3.physionet.org/)
The files are displayed if
{% project.gcp and project.gcp.sent_files %} == True
, so now we need to:# Set the sent files flag to True p = PublishedProject.objects.get(slug="kinecal", version="1.0.3") p.gcp.sent_files = True p.gcp.save()