filer
filer copied to clipboard
filerjs
Use Blob objects for storing file data
ArrayBuffer objects have the disadantage that they always have to live in memory. So if your file entries contain ArrayBuffer objects which describe the file contents, that means that the whole file contents is read into memory whenever the IDB entry is read.
However if you use Blob objects to hold the data, that means that that only the metadata about the Blob needs to be read at the time when the IDB entry is read. The actual Blob contents doesn't need to be read into memory until the Blob is read using FileReader.
Another advantage that Blob gives is that it enables the IDB implementation to store the Blob data outside of the database. This improves performance of the database since it can be kept more compact. Firefox currently does this, though I think IE does not. Not sure what chrome does since they only recently added Blob support (not sure if it's even landed yet).
The lack of Blob support in Chrome might be the main sticking issue here though...
Currently I'm using JSON objects to store the metadata, so the ArrayByffers are only read when actual data is read. Storing the data outside the database is appealing though.
One issue I've had with using Blobs is that they have their own async API, so using them might mean committing your transaction before you mean to in cases where you need data from the blob as part of a transactional operation.
I'm not very concerned about Chrome's support. As long as Blobs are in the IDB spec, I'm happy to use them and fallback to ArrayBuffers.
I'm planning on implementing blocks over objects so that writing is cheaper as well, since even though Blobs may improve read performance they are still immutable and I'd have to (I think?) write a completely new Blob each time.
Can you explain more the situation when you read to read file data as part of a transaction?
Most filesystems aren't transactional at all, other than for operations like "create file if it does not exist" or "delete directory if empty". But it's not obvious to me which atomic operations that need to involve file data.
https://github.com/eligrey/Blob.js could help us do this in cases where browsers don't support them.
Yeah, that might be the way to go. I'm considering this low proiority for now, since it mostly addresses performance.
https://github.com/nolanlawson/state-of-binary-data-in-the-browser#readme is useful as you ponder this, esp. for cross-browser support of blobs.
Don't try to store Blobs directly in IndexedDB, unless you want to cry.
Ugh, this sound like more of a headache than it's worth.