Question about pullChanges with large data.

[Acetyld]

Hello!

We have a react native app currently using redux-persist and it works good. But our app needs to work 100% offline wich works right now. But we are facing/gonna face the issue that if we add the new feature the data is to large to be in persist (memory) So currently we have are using realm in our new branch and it works fine (few bugs) and we have a useRealm wich works good. And it uses listeners just like firebase does and all in all its decent. But our problem is the community, its inactive and respond bad and it had low support for react native.

We love watermelonDB its general idea and everything except the no-hooks part (expect useDatabase). We saw some online stuff with Rxjs but its not official and not recommanded, we also found a semi-offical codebox for a start on hooks with useObersable? but this also doenst feel right.

So we are semi-almost deciding to use withObersables for now. But our biggest question is about pullChanges. We have a big Symfony + Api-Platform backend. And in the future with our new system it needs to be able to store thounands + record locally so the app can fully work offline. This works well on realm and i quess also on water melon. For realm we wanted to build a own custom sync that makes sure local is in sync with the server (only pull).

But we saw u guys have pullChanges. But our biggest concern is timeout/backend. If we call our backend with timestamp and we need to return evey change at once it can be quite huge and take a little bit of time to calculte (to say it simple we have a large task system build for campings) and everyday there can be hunders/thousands of new record. So to calculate last changes it could take a while and maybe timeout. How do you guys feel about this.

In realm we wanted to build a custom function that uses pagination to get data. And we use mercure to implement a realm time section into the app for live usage. Maybe you guys have general tips as wel regarding large local sync with react-native using watermelondb.

We have few ideas

1. Sync everyday once?
2. Make a sync button for user?
3. Sync on app open but only get tasks and other data that are 2 days old and eveything in future? Together with the following function: If you open a old task from 5 months ago it will store all data regarding to it (so only store if you opened it)

But again to stick with our main question. pullChanges with large data, how to handle if there could be a timeout or it could take long?

[radex]

Make your backend faster. In nozbe.com I have an account that has on the order of 200 000 records, and calculating changes since last sync on a simple cloud virtual instance takes on the order of 50-100ms. And there really isn't much magic there. Just use a decent database on your backend and index it properly, and you'll be fine.

Watermelon does not time out synchronize(). If you configure your network request inside pullChanges not to time out, it won't time out.

[Acetyld]

Cool! And i think its also possible to do multiple calls to synchronize right? To handel different types.

[KrisLau]

@Acetyld I believe the synchronize function is handled by your application so it can be called as many times as you deem necessary. Who calls synchronize() - WatermelonDB docs`

[radex]

Closing - question was answered