Store File Blobs in Stable Memory

[tomijaga]

Hi @ZenVoich,

I noticed that the uploaded files are currently being stored as an array of blobs in heap memory. I would like to propose a change that would store these files in stable memory once they have finished uploading, in order to take advantage of the extra 32 GB of space in stable memory.

The proposed design is as follows:

• After the file chunks are successfully uploaded to the canister, we would take the chunks from activeUploadsChunks and store them contiguously in stable memory using the ExperimentalStableMemory module.
• The location of the stored data would then be stored as a File type in a new TrieMap<FileId, File> called files.

type File = {
    offset: Nat64;
    size: Nat:
}

I believe that this change would help us make better use of the available stable memory and improve the overall performance of the system. Let me know what you think!

[ZenVoich]

Hi! Nice improvement, but currently there is a logic to spawn new storage canisters as soon as they reach the limit. And at the moment we are far from the limit for one canister(storage now takes only 10mb). Probably we can return for this feature later.

[tomijaga]

Okay, I'll return to this one later. I see that most of the new issues are assigned to you. Is it possible for me to pick up one of those issues and start working on a pr?

[ZenVoich]

Is it possible for me to pick up one of those issues and start working on a pr?

These tasks will be part of the grant application that I'm going to apply for.

There is a bug related to vessel.js if you want to take a look #25