Documentation/Questions

[thedracle]

I'm interested in knowing some details about the behavior of GroundDB.

1. When a collection has been grounded, say it's 3/4 MB, and the application disconnects, if the application is stopped and restarted, when it reconnects, does the online subscription pull the entire dataset all over again?
2. Is there some way for say Cordova, I could bundle the dataset in my application so that when it connects it only pulls down differences between the current dataset, and the subscription?

Ultimately the behavior I am seeking is to have a very large data set bundled with my mobile application, and for it to connect to the backend, and only retrieve changes, save them to local storage once, and only query for differences.

The dataset is far too large for the clients to pull every time they reconnect, or restart their application.

I'm thinking I probably will have to bundle my dataset, put it into a local collection, and make my own meteor method based mechanism to update it over the wire.

[raix]

its a very specific usecase - you could use ground db on a Mongo.Collection('foo', { connection: null }); - then have a json data set with dates on the documents. If the grounded db is empty load the json into the db.

Then have a separate collection and subscription for retrieving only the updates since the newest updatedAt/createdAt in the grounded db - on subscription ready merge the updates into the grounded db.

[thedracle]

Sounds like a great idea, thanks for the suggestions!

[malhal]

If 1 is true then that conflicts with one of the features "Resume of changes in collections". Based on that feature personally I didn't think it would have downloaded the entire collection again on re-connection. If thats the case then theres hardly any point in caching it at all. On reconnect after launching offline I'm experiencing every record being removed and re-added to the collection which seems very inefficient.

[thedracle]

So, I'm still using GroundDB for small collections that I want to remain reactive. My eventual solution has been to use PouchDB backed by Sqlite for very large collections that need to be cached on the device.

These are hundreds of megabytes, and basically contain geolocation/search data that needs to be available to my application offline.

Unfortunately this isn't ideal, and my plan is to perhaps modify GroundDB, or make something like it in the long run, to replace PouchDB (Perhaps even using PouchDB as a storage driver).

I think the ideal solution would look something like GroundDB, but would have the following additional features:

• Storage of data to Sqlite as opposed to LocalStorage (Due to the very small storage limit on mobile devices)
• The wrapped collection stores revision keys in MongoDB, and subscribes excluding revisions of data that have already been collected.

This would eliminate pulling down already cached data, and only overwrite/pull data that has modifications. The revisions would need to somehow be automatically attached when a document is updated in a collection.

It also allows you to basically bundle the SQLLite database with your application, so the first time the application starts, it can have a very sizable local database already pre-cached, and never even have to touch the server, except for small updates.

At the moment our solution is working and in production, but we have some short term plans to try to integrate it more closely into Meteor.

[raix]

@thedracle I've pushed the new code for localforage support and sqlite on cordova - it's using the upcomming ES2015 1.2 Meteor. Expect ground db to get an overhaul one of these days.

[thedracle]

@raix, Very cool, I look forward to it.

I really think GroundDB is probably the biggest missing piece for making Meteor a viable solution to creating hybrid mobile applications.

Thanks for all of the work you've done on it, it's saved me and my team a lot of time.

[thedracle]

@raix, Any details on where this new code is located so that I can check it out?