Halibut icon indicating copy to clipboard operation
Halibut copied to clipboard

Sast/async json spike in memory fallback sync

Open sburmanoctopus opened this issue 1 year ago • 1 comments

[sc-53133]

Background

Results

Before

After

To show that memory is capped, we used a large file and observed the following:

When Writing With a small memory limit, RAM didn't exceed 1.8GB image

With a very large memory limit, all data was moved into memory, reaching 2.3GB image

When Reading With a small memory limit, RAM didn't exceed 3GB image

With a very large memory limit, all data was moved into memory, reaching 5.5GB image

How to review this PR

Quality :heavy_check_mark:

Pre-requisites

  • [ ] I have read How we use GitHub Issues for help deciding when and where it's appropriate to make an issue.
  • [ ] I have considered informing or consulting the right people, according to the ownership map.
  • [ ] I have considered appropriate testing for my change.

sburmanoctopus avatar Jul 12 '23 05:07 sburmanoctopus