dataloader icon indicating copy to clipboard operation
dataloader copied to clipboard

Redundant ecto queries

Open sam701 opened this issue 4 years ago • 2 comments

I'm trying to plug this awesome dataloader module. It's working as expected so far but I'm observing redundant ecto queries in the following scenario. Here are absinthe objects of my graphql schema:

  object :topic do
    field :id, :id
    field :title, :string

    field :posts, list_of(:post), resolve: dataloader(:db)

    field :created_at, :string
    field :created_by, :user, resolve: dataloader(:db)
    field :updated_at, :string
    field :updated_by, :user, resolve: dataloader(:db)
  end


  object :post do
    field :id, :id

    field :content, :string

    field :created_at, :string
    field :created_by, :user, resolve: dataloader(:db)
    field :updated_at, :string
    field :updated_by, :user, resolve: dataloader(:db)
  end

When I fire the query below I observe 4 queries select * from users....:

query {
  topic {
   title created_by { name } updated_by { name }
   posts {
      content created_by { name } updated_by { name }
   }
  }
}

Is there a way to define the absinthe objects so that there will be just one query for the users?

EDIT: the "hack" below is wrong (see my comment). I managed to reduce the number of user queries by two using this “hack”

field :updated_by, :user, resolve: dataloader(:db, :created_by)

But is there a proper way to specify a unique key for all four fields?

sam701 avatar Jul 03 '21 08:07 sam701

The "hack" I described above is wrong. It worked only because created_by and updated_by was the same user. I will edit the issue description.

I came up with the following solution:

  defp batch_loader(source, entity, id_field) do
    fn parent, _args, %{context: %{loader: loader}} ->
      id = Map.get(parent, id_field)
      loader
        |> Dataloader.load(source, entity, id)
        |> Absinthe.Resolution.Helpers.on_load(fn loader ->
          result = Dataloader.get(loader, source, entity, id)
          {:ok, result}
        end)
    end
  end
  def user_loader(source, id_field), do: batch_loader(source, User, id_field)

and then in the schema:

    field :created_by, :user, resolve: user_loader(:db, :created_by_id)
    field :updated_by, :user, resolve: user_loader(:db, :updated_by_id)

This works. But the question now is if this can be done automatically by Dataloader.Ecto? Or are there any reasons why it can/should not be done?

sam701 avatar Jul 12 '21 11:07 sam701

The blocker here is whether Dataloader can "see through" associations on a schema to determine that they have equivalent SQL that will be run. Not sure if this is doable.

benwilson512 avatar Oct 31 '22 04:10 benwilson512