Proposal for dynamic query cache control

[tomphilbin]

trafficstars

Is your feature request related to a problem? Please describe. When running an Internet-facing graph at scale, caching is an essential tool to ensure that systems and infrastructure are only dealing with requests that they absolutely have to.

I have Hot Chocolate deployed behind an edge-cache, however the TTLs being provided to our edge are currently static and it’s essential for my use case that the cache configuration can be created dynamically based on the query and potentially other factors such as downstream API responses

Describe the solution you'd like Apollo Server has a well thought out approach to this problem in the form of cache hints that I think we could learn from.

Take the following code-first example for an ASP.NET app:

public class Query
{
    [CacheControl(1200)]
    public IEnumerable<Product> GetProductList()
    {
        // fetch and return product list
    }
}

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services
            .AddGraphQL(
                provider => SchemaBuilder.New()
                    .AddServices(provider)
                    .AddQueryType<Query>()
                    .AddCacheControl() // Field middleware to collate the CacheControl attribute values
                    .Create(),
                builder => builder
                    .UseCacheControl()); // Query middleware to chose the lowest of the collated CacheControl attribute values
    }
}

This approach could also leverage resolver context internally to provide dynamic cache control like so:

public class Query
{
    public IEnumerable<Product> GetProductList(
        IResolverContext ctx,
        [Service] ProductDataLoader loader)
    {
        var response = loader.LoadAsync();

        ctx.CacheControl.Add(response.MaxAge);

        // any other logic and return products
    }
}

Finer grain control over the header key and value could be achieved like this:

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services
            .AddGraphQL(
                // create schema
                , builder => builder
                    .UseCacheControl("Edge-Control", value => $"!no-store, max-age={value}")); // Without these arguments sensible defaults will be used, for example "Cache-Control" and "max-age={value}"
    }
}

Describe alternatives you've considered I haven't considered any other approaches but I'm open to suggestions!

Additional context This implementation is based upon a proof-of-concept that I've been working on which has the following unresolved issues:

1. I still haven't figured out how to get the values from the UseCacheControl query middleware into the ASP.NET response header (any help here would be really appreciated!)
2. I currently use "vanilla" .NET attributes for CacheControl and filter in the field middleware on context.Field.ClrType but I'm interested in whether there is a better approach, for example using custom directives.
3. The CacheControl attribute currently only accepts an integer but there could be other arguments that it should accept for use-cases that I haven't considered.
4. The current implementation only deals with pure code-first scenarios.

I'm happy to complete the necessary work for this if it's a feature that the community is interested in and welcome any help or support!

[michaelstaib]

We need to detail a bit more all the approaches, not only pure code-first. In essence, I think this will make a great addition.

Apollo also lines out a dynamic API approach where they can interact with the caching within the resolver.

What we need to work out:

• schema first => directive
• code-first => fluent API
• pure code-first => descriptor attribute

Necessarily all those different approaches will translate into the same mechanism.

[michaelstaib]

https://www.apollographql.com/docs/apollo-server/performance/caching/#defining-cache-hints

[michaelstaib]

@rstaib fyi

[michaelstaib]

Also what should we use underneath? the standard Microsoft caching APIs?

[tomphilbin]

Thanks for the feedback!

I thought it would be worth writing the documentation for this first in order to better flesh out the API (and later, get free documentation 🥳)

Rather than clutter up the comments here's a link to the draft: https://demo.codimd.org/csYj21OST6eHQJHIfAvvsg?view

I definitely think we should use MSFT APIs underneath as they're quite comprehensive.

[rstaib]

Perhaps we should also elaborate other approaches to see which suites more, and to see what are the pros and cons. I think the approach which apollo is following is not bad by the way.

[michaelstaib]

@tomphilbin documentation looks already good. I have given you write access on the hc repo. So you can create your own branches on this repo. Rafi can help you integrating the documentation into the hc documentation on this repo. Great work so far.

[michaelstaib]

@tomphilbin shall we discuss the on the next working group meeting? I am keen to move forward on this but want to discuss the how a bit.

[tomphilbin]

Yeah sounds good, is it on Tuesday?

[PascalSenn]

great work @tomphilbin ! I wish every feature request would look like this! This got me thinking today. Together with execution plan this could become even more powerful. If we annotate fields we could even cache whole queries and offload them to a cache. The execution engine could work out what the best caching strategy for a query is and generate a cash key based on variables and query. The whole cache could be abstracted with a simple api for storing, fetching and invalidating these keys. It would be easy to create providers for this API. This way we could also integrate things like redis easily. Maybe we could even invalidate the cash with mutations.

[tomphilbin]

Thanks @PascalSenn! I think your suggestion sounds great and I'm super keen to get started on this. Is there any chance you and/or others would be available for a quick call to discuss an approach in more depth? I'll document any outcomes in this thread.

[tobias-tengler]

I'm really interested in adding HTTP caching capabilities, so I would like to give this a jab in the future. I really like the Apollo approach in this regard and would try to port their @cacheControl functionality to Hot Chocolate.

[michaelstaib]

Sure, shall we do a zoom?

[michaelstaib]

I have put it on the backlog for 13.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.