Relation Efficiency Solutions

[jacobaclarke]

I am trying to make a very large request that does lookups on several different collections to build a large graph. This ends up being very slow because the data has to be sent between my server and database several times to make the request.

It is pretty simple to add lookups to an object in mql and save the combined graph as a view. However, I can't access the combined types from graphql-compose-mongoose. Is there any way to create views from graphql-compose-mongoose that I can then use as type composers?

[jacobaclarke]

Here is a simple example of what I would like to do:

Models:

users {
	eventIds
}
events {
	name
	extraRandomFieldIDontWantProjected
}

Then I would create a view from this aggregation

Users.aggregate[
{ $lookup:
	  {
	    from: 'events',
	    localField: 'eventIds',
	    foreignField: '_id',
	    as: 'events'
	  }

}
]

Then I could make a graphql request like this on the newly created view

users {
	events {
		name
	}
}

I understand that this same functionality exists on addRelation, but if I understand correctly, the addRelation approach requires multiple requests to the datbase, whereas this would require one.

Thanks a ton in advance. I really appreciate any input you have.

[jacobaclarke]

It looks like your implementation of dataloader may do this, but I am having a hard time understanding it. From my understanding, it caches the _id index from the last request so that repeated lookups don't need to make a trip to the database. My use case is below:

Resolvers.StudentTC.addRelation("clinics", {
  resolver: Resolvers.ClinicTC.mongooseResolvers.findMany(),
  prepareArgs: {
    filter: (source) => ({
      isActive: true,
      positions: { signUps: { studentId: source["_id"] } },
    }),
  },
  projection: { ["_id"]: 1 },
} as any);

This doesn't seem to work with the implementation because studentId on the foreign document is a nested subdocument and dataLoader requires that the foreignKey be on the main document. Am I understanding this right?

Sorry for all the questions. This repo is a great tool!

[nodkz]

@j718 you have a complex issue that is not covered by graphql-compose-mongoose code (resolver) generators. And frankly, it really quite complex or maybe impossible to build the out-of-the-box solution.

My recommendation is to write your complex resolvers manually. You should write some Service or DataManager which will fetch data from database in performance way via aggregation or any other way. And after that from your manual written resolvers just call your Service (DataManager).

UserTC.addField({
  events: {
    type: EventTC.List,
    resolve: (userData, args, context, info) => {
      return yourEventServiceWithCustomDataLoaderImplementationForNestedField.loadMany(userData.eventIds);
    }
  }
})

[jacobaclarke]

Thanks for the feedback. I'll try that out.

[jacobaclarke]

@nodkz Is there any difference in efficiency between adding a relation through a mongoose virtual vs the TC.addRelation method?

[nodkz]

addRelation has a small runtime overhead in args preparation https://github.com/graphql-compose/graphql-compose/blob/e7467d505c765beeec6d53f17a608270545daef6/src/ObjectTypeComposer.js#L1783-L1798

But it so small that we can neglect it.

[yossisp]

Perhaps, you need to write your custom dataloader in order to batch requests, if so I wrote an example here which uses graphql-compose-mongoose

[jacobaclarke]

Looks like a great post. Thanks for sharing! I'll try implementing this