PowerApps-Tooling icon indicating copy to clipboard operation
PowerApps-Tooling copied to clipboard

oom on msapps with large files

Open MikeStall opened this issue 4 years ago • 1 comments

The basic problem is that we load the entire msapp into memory (CanvasDocument).

And then oom at places like this:

                        case FileKind.AppCheckerResult:
                            var appChecker = Encoding.UTF8.GetString(entry.ToBytes());
                            app._entropy.AppCheckerResult = appChecker;
                            break;

We should calculating this in a streaming way rather than ToBytes(). (sarif file can be huge)

We should scrutinize usage of FileEntry.ToBytes, and probably need to be lazier and have documents backed by disk.

MikeStall avatar Jan 06 '21 00:01 MikeStall

This is a broader problem ... we load all files into memory for CanvasDocument. Sarif is one example (and we load that into entropy), but we track all the other files too:

internal Dictionary<string, FileEntry> _unknownFiles = new Dictionary<string, FileEntry>();

internal Dictionary<string, FileEntry> _assetFiles = new Dictionary<string, FileEntry>(StringComparer.Ordinal);

For example, we might hit this problem with large resource or asset files. Screens and connections are smaller and probably not an issue.

MikeStall avatar Jan 06 '21 18:01 MikeStall