insertMany support

[hwillson]

Migrated from: meteor/meteor#8377

[dr-dimitru]

1. On Server it will work via rawCollection
2. @msavin not sure, Meteor has its own mongo API flavor, NPM mongo package and Meteor's mongo too different, updating it to meet API of NPM package will break all apps. For example .insert() is deprecated in favor of .insertOne() and .insertMany()

[edemaine]

@dr-dimitru rawCollection doesn't do the automatic _id work that Meteor does. I've tried using it, but it was definitely awkward. Anyway, I don't see why Meteor shouldn't follow modern MongoDB interfaces when they are natural extensions of what we already support (which were just designed on old MongoDB interfaces).

[dr-dimitru]

@edemaine

Anyway, I don't see why Meteor shouldn't follow modern MongoDB interfaces when they are natural extensions of what we already support (which were just designed on old MongoDB interfaces).

Compatibility. It's either should be upgraded to use latest MongoDB API behind the scenes, or cause all developers to rewrite most of MongoDB calls.

I'll vote for "magic" behind the scenes, for example when .insert({Object}) will trigger .insertOne(), and .insert([{Object}, {Object}]) will trigger .insertMany(). So none of us will have to update code, and get all advantages of latest NPM MongoDB driver.

[edemaine]

That's how the modern MongoDB API works. It is backwards compatible. You can specify a single document or an array or documents to insert, and insertMany and insertOne are just variations where you are asserting that the argument is one way or the other. It's exactly what you'd hope for!

[dr-dimitru]

@edemaine you're right, but it has deprecation notice, and methods may be removed on any major update. I mean Meteor's mongo db definitely needs an upgrade, but there is much more than just adding .insertMany and requires a lot of work

[edemaine]

Ah, I missed the deprecation. I agree that a warning be annoying in Meteor, so we should just magically call the right function.

Anyway, besides API, the bigger issue here is supporting insertion of multiple documents in one request to the server.

[dr-dimitru]

Anyway, besides API, the bigger issue here is supporting insertion of multiple documents in one request to the server.

D you mean from the Client?

[edemaine]

I meant one call to the MongoDB server from the Meteor server. I'm guessing that this is less of an issue for MiniMongo on the client, though it'd be nice to have the same interface for consistency.

[dr-dimitru]

Sure, we are on the same page

[amanagarwal2189]

One year hence, has this feature been introduced?

[smeijer]

A more up to date mongo interface would be most welcome. I've migrated my app to use (mostly) only rawCollections, because of this reason.

Now I have promises (await collection.find(...)), insertMany, aggregate, and all other nice mongo stuff.

[StorytellerCZ]

Probably the path of least resistance is to update the insert function to accept array of objects and if it gets that for it to use insertMany and in case of a simple object, just use insertOne. From my preliminary investigation that doesn't look as much of a problem, the issue comes when dealing with mini mongo. I'll try to find some time to look into this.

[msavin]

Hmm yes but I think that kind of thing would require to make Meteor to make multiple requests to the database, whereas insertMany should work in one request.

On Mon, Nov 5, 2018 at 11:16 AM Jan Dvorak [email protected] wrote:

Probably the path of least resistance is to update the insert function to accept array of objects and if it gets that for it to use insertMany and in case of a simple object, just use insertOne. From my preliminary investigation that doesn't look as much of a problem, the issue comes when dealing with mini mongo. I'll try to find some time to look into this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/meteor/meteor-feature-requests/issues/7#issuecomment-435823073, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7-g26rlf7feMBEc7rSWx3o4a7y8qU6ks5usA_5gaJpZM4NxyXc .

[StorytellerCZ]

I'm thinking:

if (multi) {
  const result = this._callMutatorMethod("insertMany", doc, wrappedCallback);
} else {
  const result = this._callMutatorMethod("insertOne", doc, wrappedCallback);
}

We need to handle all the other Meteor stuff before this and then inserting into minimongo (for proof of concept just an loop around the current method might suffice).

[smeijer]

Sounds like a solution to me. And I do understand that we need to start somewhere, but I would advise to also look at the bigger picture. There are more Mongo things that Meteor dev's do not have access to, without depending on the rawCollection method.

I think even the issue behind this future requests, requests for a bigger change than the title of this issue:

https://github.com/meteor/meteor/issues/8377#issue-207887730

Mongo 3.2 seems to have moved to a new naming model: insertOne, insertMany, and updateOne, updateMany (replacing the multi flag). It would be nice if Collections interface could support this alternative too.

I don't know how fast mongo will move on this, but note that the methods without the One or Many suffix have been deprecated.

insert => use insertOne, insertMany or bulkWrite
update => use updateOne, updateMany or bulkWrite
remove => use deleteOne, deleteMany or bulkWrite

count => use countDocuments or estimatedDocumentCount instead

http://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html

There is more that's been deprecated, for example the fields property in the find options should be replaced by projection.

But, it could be that I'm talking nonsence here, and that meteor/mongo is already being refactored behind the screens to translate old calls to new ones. In that case, feel free to ignore me 😇

[StorytellerCZ]

@smeijer Sadly not, this is an issue as far as I can see. I think the for these One/Many calls that can be done relatively painlessly. I think the issue comes with minimongo where we need to replicate a lot fo the functionality from MongoDB.

[StorytellerCZ]

Made a quick dirty concept: https://github.com/StorytellerCZ/meteor/blob/mongodb-one-many/packages/mongo/collection.js (not even tested yet :sob: ).

[SimonSimCity]

@smeijer you mean

https://github.com/meteor/meteor/blob/24865b28a0689de8b4949fb69ea1f95da647cd7a/packages/mongo/oplog_observe_driver.js#L89

and

https://github.com/meteor/meteor/pull/9942#discussion_r218564879

...

Meteor in fact uses a quite recent version of the MongoDB driver, but backports a lot of the functionality because it introduced the new version of the MongoDB driver in a release where it shouldn't break compatibility (at least not introduce the big BC breaks).

[smeijer]

The projection, indeed. Nice one.

The second link though; not sure how I should read that without diving in the code:

https://github.com/meteor/meteor/commit/2a8dea89349935ff0e0452e5d1a1f5b09df954ad

Silence deprecation warnings introduced in a mongodb patch update. These deprecation warnings were introduced in [email protected]:

It doesn't sound like a long term fix. But again; I haven't read the code, and I can be wrong.

[SimonSimCity]

It removes the deprecated message added when using insert instead of insertOne or insertMany. 😅 Sure this will back-fire the one or other day since these functions will be removed in the long run ... and we should be prepared for it.

[smeijer]

That was exactly what I was thinking after a quick look, but was really hoping I missed something there.

Let's hide deprecation messages.

Sounds like a solid plan 😅.

[klaussner]

The mongodb version that adds the deprecation messages also includes an important bugfix and was published shortly before the Meteor 1.8 release. The workaround in https://github.com/meteor/meteor/commit/2a8dea89349935ff0e0452e5d1a1f5b09df954ad isn't meant to be a long-term solution. :slightly_smiling_face:

[SimonSimCity]

There's a PR in progress for this https://github.com/meteor/meteor/pull/11200 🎉

[StorytellerCZ]

For the latest version see: https://github.com/meteor/meteor/pull/11222