physionet-build icon indicating copy to clipboard operation
physionet-build copied to clipboard

Allow published_project.gcp.sent_files to be toggled (True/False) via the admin console

Open tompollard opened this issue 2 months ago • 1 comments

For large files, we often have to manually transfer the data to GCP using the approach outlined below. Either we:

  1. improve the process for transferring files to the cloud, so that it doesn't fail for large datasets or
  2. add a form field to the published-projects page in the console that allows the published_project.gcp.sent_files flag to be toggled.

Add data to Google Cloud bucket

First go to the project management page (e.g. https://physionet.org/console/published-projects//kinecal/1.0.3/) and click “Create bucket” to create a bucket on Google Cloud.

Now the bucket has been created, we can upload the files directly from the PhysioNet server using gsutil:

gsutil -m cp -r ./* gs://[kinecal-1.0.3.physionet.org](http://kinecal-1.0.3.physionet.org/)

The files are displayed if {% project.gcp and project.gcp.sent_files %} == True, so now we need to:

# Set the sent files flag to True
p = PublishedProject.objects.get(slug="kinecal", version="1.0.3")
p.gcp.sent_files = True
p.gcp.save()

tompollard avatar Apr 24 '24 15:04 tompollard