Mirabuf Caching

[a-crowell]

Description

Utilizing browser storage for mirabuf caching: local storage for a map of keys and metadata, and OPFS for storing mirabuf data

Objectives

• [X] Cache from fetchLocation or ArrayBuffer (if not already in)
• [X] Get assembly from id
• [X] Remove specific entry from local storage and OPFS
• [X] Clear storage

Testing

Two new temp buttons on the MainHUD:

• Print Mira Maps -> prints the maps of robot/field fetch locations and ids in console
• Clear Mira -> clears both these maps stored in local storage and any mirabuf files stored in OPFS

Other than making sure robots and fields import correctly (no user difference from before), you can also check:

• Pulling from cache when it can and remote when it needs to (via console "Loading mira from cache" vs "Loading and caching new mira")
• Saving correctly (via "Print Mira Maps" for the local storage maps and an OPFS explorer for the mirabuf files)

[!Note] A DOMException for modifying unmodifiable shows up upon clearing mira, specifically the OPFS. None of the methods I have tried have avoided this error, but it still clears the OPFS despite it. I'm still exploring it, so let me know if you have any ideas.

[!Note] Currently dealing with npx issues, so not formatted for now.

JIRA Issue

[autodesk-chorus[bot]]

Chorus detected one or more security issues with this pull request. See the Checks tab for more details.

As a reminder, please follow the secure code review process as part of the Secure Coding Non-Negotiable requirement.

[HunterBarclay]

@BrandonPacewic Could you commit those fixes?

[BrandonPacewic]

@BrandonPacewic Could you commit those fixes?

@HunterBarclay, committed.

[a-crowell]

@BrandonPacewic I have never seen that before. 1. What were the steps you took before then and 2. When was the last time you cleared browser history?

[BrandonPacewic]

@BrandonPacewic I have never seen that before. 1. What were the steps you took before then and 2. When was the last time you cleared browser history?

@a-crowell

Steps:

• Loaded synthesis normally
• Printed the mira maps (normal behaviour)
• Spawned the field
• Printed the maps again (detailed behaviour)

I don't think I have ever cleared my browser history on this computer, what affects are we expecting / accounting on from that?

[a-crowell]

Steps:

• Loaded synthesis normally
• Printed the mira maps (normal behaviour)
• Spawned the field
• Printed the maps again (detailed behaviour)

I don't think I have ever cleared my browser history on this computer, what affects are we expecting / accounting on from that?

@BrandonPacewic

If the cache hadn't been cleared, the blob could have been leftover from an old review. But the maps printed empty just a few seconds earlier?

[a-crowell]

Out of ideas for the localhost blob in local storage and unable to recreate it on any laptops (mine, Brandon's, Dhruv's) even checking for first time running this code, so leaving be for now.

[HunterBarclay]

@BrandonPacewic I used a URL.createObjectURL() for the ImportLocalMirabufModal to convert the blob I get from the file input to a downloadable URL. We should probably customize the key for those, maybe a hash of the blob?

[HunterBarclay]

@a-crowell ^^^

[PepperLola]

What's the purpose of hashing the mira data?

[a-crowell]

@PepperLola The caching usually compares the url/fetchLocation to determine whether or not the file is in the cache already. The ImportLocalMiraModal uses createObjectURL to pass in a fetchLocation though, giving keys of "blob: ... localhost ...", so instead, we now use the hash as a key.

[PepperLola]

@PepperLola The caching usually compares the url/fetchLocation to determine whether or not the file is in the cache already. The ImportLocalMiraModal uses createObjectURL to pass in a fetchLocation though, giving keys of "blob: ... localhost ...", so instead, we now use the hash as a key.

If you have to load, unzip, and decode the mira to hash it before checking if it's already in the cache, wouldn't that defeat the point of caching it in the first place? Or is there something I'm missing?

[a-crowell]

If you have to load, unzip, and decode the mira to hash it before checking if it's already in the cache, wouldn't that defeat the point of caching it in the first place? Or is there something I'm missing?

You're right what is the point of this.

In case Hunter meant hash of the url, I tried that too, but the whole thing is that it's different every time so it doesn't help either.

The two options I see are to either use the file name or just accept a new file from that modal and not cache it. @HunterBarclay You were against cutting down the fetchLocation to the name for other files--do your reasons still apply here?

Is there another way to check for same files I'm not thinking of?

[a-crowell]

@PepperLola Thanks for pointing that out! I removed the remote version since it was never used anyway and adjusted CacheAndGetLocal to avoid grabbing from OPFS while taking advantage of having the assembly already to cache the name.

[a-crowell]

@BrandonPacewic Yep, all covered! That modal now uses a hash of the file as key instead of url, so you won't find another localhost in the map.

[HunterBarclay]

Fixed the unit tests and added a few clean ups. Didn't format because it still doesn't work on my desktop.

[HunterBarclay]

Merging for now. This Prettier issue is going to be a continuous problem