Meteor-CollectionFS Handling offline files (audio/video) in cordova

Hi,

I'm building a meteor and phonegap based app for IOS. I have mp3 files on the meteor server stored with collectionFS (also tried other stuff). Everything works fine :)

I'm now trying to get the app to work offline. Do you have any hints what the best way would be to get the files to be persisted locally via phonegap within the app?

I was thinking of using the Cordova File API

Do you have any other / better ideas on how to do this?

Cheers, Gerwin

Jul 09 '14 16:07 gerwinbrunner

Its on the roadmap - Milestone 1.7.0 https://github.com/CollectionFS/Meteor-CollectionFS/issues/215

The way we eventually want this to work would involve some features helping with quotas in the client environment.

We need the client-side streams to align / simplify the client-side api - at the moment we only support file transport via http, but would like to use ddp or websockets too. (we got ddp but its slooow)

Having client-side files will be used for stuff like latency compensation, background uploading, auto resume and offline access.

For now I'd simply fetch the data and store it in local filesystem - I would use something like GroundDB to persist my db - in the db put a reference to the file - I'd use the blob reference not put the whole base64 url in img src (saves memory) - I'd have a fallback in case the user started the app on an empty phone.

The file reference could look something like: (with cfs-ejson-file installed)

  doc.picture = {
    local: 'file://',
    remote: myCfsFile // FS.File is converted to EJSON reference
  };

Ideally we would have a way to couple a local storage adapter to a collection fs something like:

  // Client
  var Images = new FS.Collection("images", {
    stores: [new FS.Store.CordovaFileSystem("images", {
      path: "file://uploads",
      transformWrite: function(fileObj, readStream, writeStream) {
        // Transform the image into a 10x10px PNG thumbnail.
        // We must change the name and type, but the new size
        // will be automatically detected and set.
        fileObj.extension('png', {store: "thumbs"});
        fileObj.type('image/png', {store: "thumbs"});
        gm(readStream).resize(60).stream('PNG').pipe(writeStream);
      }      
    })]
  });

But what about quota limits and app resets etc.? So File.url() would know if the file is locally or it should go fetch it remotely - if it got a the CordovaFileSystem attached it would actually download the file to local and then return the local reference. If it havent got space enough to store the file it would request space and doing so would remove some local files to free up space. We would need a general api to describe the conflict resolution eg. finding the files to remove, the oldest or the less important etc.

Just some quick thoughts / feedback - you are welcome to contribute by prototyping/share ideas/needs in this area trying to find patterns / antipatterns - we have a private prototype of some of this. eg. GM like api and "latency compensation".

So why bother having a GM like api on the client? Well the client-side storage adapters has a thing to say about how the data is uploaded/downloaded etc. and its part of being able to upload directly to S3 etc. Cordova can reduce image size for us, but we may want to upload multiple sizes etc.

But the biggest missing piece is client-side streams - and thats the part I'm working at the moment when I got time + test writing.

Jul 09 '14 16:07 raix

Thanks so much for that answer!

I'm already using GroundDB to persist data on the native app. Works nicely. I'll try to get the files stored in an client only collection later and have a reference to the file and see how this works for me.

The App has a similar workflow as Audible or the Kindle. So you decide to "install" stuff on your device. Which means that your suggested solution should work out fine.

Your idea of the "advanced" version sounds really awesome. it would make it possible to use part of the content right away without having to wait till all is downloaded by using the server side stuff. Very awesome! If there is something you want me to test out, let me know. I have iDevices and aDevices lying around for testing :)

Jul 10 '14 16:07 gerwinbrunner

Hi @gerwinbrunner, any success on caching images locally for offline use in an mobile app?

I'm just facing the exact same situation, already using GroundDB for caching data, but wondering how to get images cached as well with automatic fallback to the remote server.

Thanks