iaso icon indicating copy to clipboard operation
iaso copied to clipboard

SLEEP-1337 Add support for mobile bulk uploads

Open bramj opened this issue 1 year ago • 1 comments

In Trypelim, we have performance issues on the form uploads. To improve the performance of the uploads, we're grouping forms and org units into zip files (batches of 2000 forms) and upload these with only 1 API call per zip file.

This PR adds a job to process these bulk uploads:

  1. A new API endpoint /api/mobile/bulkupload/ that takes a zip_file. This endpoint simply uploads the file to S3, launches an async job, and returns HTTP 204.
  2. The process job creates a record in the APIImport for future reference (and to be able to tag as has_problem) and processes all files in the zip in a db transaction.

Related JIRA tickets : https://bluesquare.atlassian.net/browse/SLEEP-1337

Self proofreading checklist

  • ~~[ ] Did I use eslint and black formatters~~
  • [x] Is my code clear enough and well documented
  • ~~[ ] Are my typescript files well typed~~
  • [ ] New translations have been added or updated if new strings have been introduced in the frontend
  • [x] My migrations file are included
  • [x] Are there enough tests
  • [ ] Documentation has been included (for new feature)

image

bramj avatar Apr 22 '24 13:04 bramj

@mathvdh assigning you for review, I think Martin is too busy at the moment

bramj avatar May 14 '24 09:05 bramj