Bulk insert into Realm

[squirtle1990]

Goals

Inserting collection of realm models into Realm

Expected Results

insertOrUpdate like Realm Java 1.1.0+ (https://realm.io/blog/realm-java-1-1-0/)

Actual Results

Can only insert new objects one at a time.

Steps to Reproduce

See above

Code Sample

realm.write(() => {
                        realm.create('ObjectName', {
                            foo: foo, baz: baz
                        })
                    })

Version of Realm and Tooling

• Realm JS SDK Version: 2.15.3
• Node or React Native: Node
• Client OS & Version: macOS 10.13.6

[nirinchev]

Do you have a reason to believe the performance of inserting objects one at a time is not sufficient for your use case?

[squirtle1990]

I am inserting tens of thousands of objects, which takes a decent while to process.

It's strange java received this feature last year, and javascript does not have it after all this time.

[nirinchev]

It hasn't been requested. What's the use case of inserting tens of thousands of objects on a user device? If these are the same for each user, maybe you can prepopulate the Realm and bundle it in your app. Even with the bulk insert that Java has, it is unlikely that this will take unnoticeable time.

[squirtle1990]

I should clarify, this is a server-side node environment. I am inserting new data into the realm DB and client apps download a subset of this data.

[bmunkholm]

@ogtimothymiller As long as you insert all your objects in the same transaction that should make it as fast as possible. It may be that other smaller things can be optimized, but the biggest gain is one transaction. I've added a "Help Wanted" if anyone is interested in exploring optimization as it can be done entirely in this repo.

[HughBerriez]

@nirinchev Am I missing something? I'm working on a react native project for a company that wants to persist 35,000+ objects locally (Android focused, at present). It seems the Realm Javascript api, Realm.create takes 47-50 seconds to write 1000 objects. Is there a sped up realm.insert* api ready and available to possibly speed things up? If not, I'll sadly have to make the decision to switch over to SQLite as our localDB solution. Hopefully I'm missing something - as the Java and Swift apis are fully fledged and seem to not have this problem.

[bmunkholm]

@HughBerriez It shouldn't be that slow. Are you doing individual transactions or one big transaction? If you want us to look closer at it, please create another issue, preferably with the code you use to insert.

[HughBerriez]

@bmunkholm I'd be happy to - but before I do spend the time, am I misunderstanding the following realm.create() being nested within a realm.write() -am I misunderstanding realm.create? And if I am, is begin/commitTransaction faster? And if so, I've tried an implementation of that, and it was just as slow - but maybe I'm implementing the code wrong? If you think that's the case, may I get a quick code snippet on the correct usage? Thank you for the time & support!

ie.

try {
 //Items is an array of objects. The ListingsAllCall in the realm.create method is the exact schema to be mapped to
 const { items } = data;

 realm.write(() => {
   console.log('ListingAllCall map to realm start.');

   // The following takes 47 seconds to complete.
   items.forEach(obj => {
     realm.create(ListingsAllCall, obj);
   });
 });

 let t1 = performance.now();
 console.log(`Finished writing in ${(t1 - t0) / 1000} seconds...`);
} catch (err) {
 console.log(err);
}
};

[cmelchior]

The above code looks fairly correct. write() will automatically begin/commit the transaction. How much data do your objects contain. Are relatively lightweight or do they contain a lot of properties or references to other objects?

[HughBerriez]

@cmelchior @bmunkholm I've created an issue to dive deeper. Thanks for the support - please ref #2154

[maxammann]

The reason for this slow performance is probably an enabled debugging mode (with Chrome e.g.)