dataloaden icon indicating copy to clipboard operation
dataloaden copied to clipboard

Can disable caching?

Open KingWu opened this issue 3 years ago • 10 comments

KingWu avatar Sep 10 '20 17:09 KingWu

Just have same question, could I disabled caching because it'll cause an issue when we change data

UPDATED: You can use clear cache by this method

func (r *shipmentResolver) Customer(ctx context.Context, obj *model.Shipment) (*model.Customer, error) {
	customer, err := dataloader.For(ctx).CustomerByID.Load(convertedID)
        // Clear after loaded it
	dataloader.For(ctx).CustomerByID.Clear(convertedID)
	if err != nil {
		// Handle Error
	}
	return customer, nil
}

duong-se avatar Oct 23 '21 14:10 duong-se

@tanphamhaiduong can I ask you why Clear() is needed?

frederikhors avatar Oct 23 '21 14:10 frederikhors

@tanphamhaiduong can I ask you why Clear() is needed?

Because some entity i need to update frequent, so clear cache to make sure it return correct value for the user

duong-se avatar Oct 23 '21 17:10 duong-se

Ok. But I still don't get it. Are you updating that entity frequently during that call?

frederikhors avatar Oct 23 '21 17:10 frederikhors

Need to be able to disable caching. Causing issues for us immediately right after deploy. User changed its name, but the old user name is getting loaded. Not good. I would don't need caching at all. It's causing more problems than it solves.

sandorvasas avatar Jan 10 '22 15:01 sandorvasas

I am unsure why caching is even implemented at all. Caching should and always should be controlled by the developer. If i wanted to add caching I would have implemented it in the data loader step, when i fetch the data from the database, ie checking redis or a in memory cache. I am likely going to have to fork to remove this functionality and it this feature should be seriously reconsidered.

TroyKomodo avatar Jan 19 '22 07:01 TroyKomodo

The caching is intended to work per request and not to be long-lived. To make sure this works properly you have to return a new dataloader for every request .

ldrs = Loaders{
Todos:  newTodoLoader(),

func newTodoLoader() *TodoLoader {
	return &TodoLoader{
		wait:     wait,
		maxBatch: 1000,
		fetch: func(keys []ID) ([]*Todo, []error) {
                         // ... retrieve data
			return todos, errors
		},
	}
}

Also make sure you are using the context as in the example folder.

TuringJest avatar Mar 05 '22 07:03 TuringJest

@TuringJest this is not correct.

  1. Why would you want dataloading to be scoped to a single request when you can scope it to the entire application?
  2. If I re-query the dataloader you would assume that the function is rerunning the database queries, if I wanted to implement caching, at a request or app scope, I would have done it in my own code. Limiting the dataloader to be used in this way is quite strange.

TroyKomodo avatar Mar 07 '22 13:03 TroyKomodo

The caching that happens on a request level doesn't stop you to use your own global caching solution to retrieve data inside the dataloader.

It just ensures that the data returned in one request is consistent in all cases and not re-queried if requested multiple times across one request.

But you can check https://github.com/graph-gophers/dataloader which offers to disable caching completely.
If you want to use this with gqlgen you can also find an example in the documentation of the master-branch.

TuringJest avatar Mar 08 '22 18:03 TuringJest

@TuringJest I think you miss-understood.

There is no way to have a global data loader without caching, your proposed solution to get around the caching of recreating it per request is not global. I fixed this problem by forking it and removing the caching aspects. Caching here is stupid, because every application has different requirements and different needs, perhaps a better way to approach this is to allow a ARGV flag which disables the caching code generation, although I didn't do this because I don't really need caching at all.

This library is better than the graph-gophers lib because this is a code generator which means all the logic is run and confirmed at compile time (which is better) than running and checking logic at runtime (which is error prone). They both do data loading but generally being able to force types at compile time is far superior than forcing them at runtime.

TroyKomodo avatar Mar 08 '22 18:03 TroyKomodo