[SPIKE] Investigate upper bound of CSV export features
Background
With the new data retention policy, we need to enable users to export their test and patient data. Back of the napkin product usage shows us that
- the maximum test results download size is ~88mb for one facility
- the maximum patients download size is ~17.5mb for one facility
The question is - if we want to enable facilities to export their data, can we support direct download? Or does there need to be an async experience?
Ask
Investigate if we can support direct downloads to help us decide if we'll need to build an async download experience instead. Please check the behavior at these thresholds:
- 100mb (upper bound with buffer)
- 88mb (expected upper bound)
- 20mb (98% of all facilities fall under this threshold for test data. All patient data falls under this)
- 10mb (97% of all facilities fall under this threshold for test data. 98% patient data falls under this)
There is an existing test data download feature that you can use to test.
Additional context
- Product brief
- Product POC: @mindyatwork
I confirmed that the upper bound numbers are still accurate and don't need to change.
Please see the Spike doc for this ticket.
Overall Conclusion: An async process will be required to address files larger than 10 mb
Thanks @arinkulshi-skylight ! I reviewed the doc. From my perspective, this ticket is good to close. I'll take this finding back and think about next steps.
Next steps:
- Understand if we can increase the timeout
- See if we can easily compress the files before download.