Deferred Uploading

[JeffreyThiessen]

Describe your idea for a new feature

Alternative way of uploading sequence files. Basic premise:

• Specify which files / samples will be uploaded to IRIDA, and create dummy/empty files which are hidden from the users.
• An outside process moves the actual files to the location IRIDA generated the dummy files.
• Once finished, files are unhidden

This would allow fast uploading when the sequence files already exist on the same drive/network, as we could take advantage of file transfer speeds that don't rely on HTTP/REST

Proposed code flow for uploading in this way

Directory of new run discovered Validate Run

• Existing uploader code
  • Files exist and well formatted
  • check projects to upload to exist

Create Sequencing Run (IRIDA)

• Existing API endpoint (Python API)

Set Sequencing Run to UPLOADING

• Existing API endpoint (Python API)

Create Samples on Projects

• Existing API endpoint (Python API)

Create empty files with matching filenames

• New API Endpoint Needed

IRIDA Needs to have these files hidden

• While status is not COMPLETE
• New functionality Needed

Transfer real files to temporary empty files

• New Integration functionality

Set Sequencing Run to COMPLETE

• New functionality needed to show files
• Ensure Analysis does not run until set to COMPLETE
• Maybe need to show on sample that files are being processed so user is not confused by empty samples existing in their IRIDA projects?