How Can We Simplify Creating Connections

[RehanSaeed]

In my .NET Boxed GraphQL project template I have an example of creating a GraphQL connection. I feel like there is too much boilerplate you have to write to get one of these working and I'm wondering if there are some simple ways we can simplify their creation by providing some helper methods.

Connection

Here is the main code to create the connection on my query graph type:

public class QueryObject : ObjectGraphType<object>
{
    private const int MaxPageSize = 10;

    public QueryObject(IDroidRepository droidRepository)
    {
        this.Name = "Query";
        this.Description = "The query type, represents all of the entry points into our object graph.";

        this.Connection<DroidObject>()
            .Name("droids")
            .Description("Gets pages of droids.")
            // Enable the last and before arguments to do paging in reverse.
            .Bidirectional()
            // Set the maximum size of a page, use .ReturnAll() to set no maximum size.
            .PageSize(MaxPageSize)
            .ResolveAsync(context => ResolveConnection(droidRepository, context));
    }

    private async static Task<object> ResolveConnection(
        IDroidRepository droidRepository,
        ResolveConnectionContext<object> context)
    {
        var first = context.First;
        var afterCursor = Cursor.FromCursor<DateTime?>(context.After);
        var last = context.Last;
        var beforeCursor = Cursor.FromCursor<DateTime?>(context.Before);
        var cancellationToken = context.CancellationToken;

        var getDroidsTask = GetDroids(droidRepository, first, afterCursor, last, beforeCursor, cancellationToken);
        var getHasNextPageTask = GetHasNextPage(droidRepository, first, afterCursor, cancellationToken);
        var getHasPreviousPageTask = GetHasPreviousPage(droidRepository, last, beforeCursor, cancellationToken);
        var totalCountTask = droidRepository.GetTotalCount(cancellationToken);

        await Task.WhenAll(getDroidsTask, getHasNextPageTask, getHasPreviousPageTask, totalCountTask);
        var droids = getDroidsTask.Result;
        var hasNextPage = getHasNextPageTask.Result;
        var hasPreviousPage = getHasPreviousPageTask.Result;
        var totalCount = totalCountTask.Result;
        var (firstCursor, lastCursor) = Cursor.GetFirstAndLastCursor(droids, x => x.Created);

        return new Connection<Droid>()
        {
            Edges = droids
                .Select(x =>
                    new Edge<Droid>()
                    {
                        Cursor = Cursor.ToCursor(x.Created),
                        Node = x
                    })
                .ToList(),
            PageInfo = new PageInfo()
            {
                HasNextPage = hasNextPage,
                HasPreviousPage = hasPreviousPage,
                StartCursor = firstCursor,
                EndCursor = lastCursor,
            },
            TotalCount = totalCount,
        };
    }

    private static Task<List<Droid>> GetDroids(
        IDroidRepository droidRepository,
        int? first,
        DateTime? afterCursor,
        int? last,
        DateTime? beforeCursor,
        CancellationToken cancellationToken)
    {
        if (first.HasValue)
            return droidRepository.GetDroids(first, afterCursor, cancellationToken);
        else
            return droidRepository.GetDroidsReverse(last, beforeCursor, cancellationToken);
    }

    private static async Task<bool> GetHasNextPage(
        IDroidRepository droidRepository,
        int? first,
        DateTime? afterCursor,
        CancellationToken cancellationToken)
    {
        if (first.HasValue)
            return await droidRepository.GetHasNextPage(first, afterCursor, cancellationToken);
        else
            return false;
    }

    private static async Task<bool> GetHasPreviousPage(
        IDroidRepository droidRepository,
        int? last,
        DateTime? beforeCursor,
        CancellationToken cancellationToken)
    {
        if (last.HasValue)
            return await droidRepository.GetHasPreviousPage(last, beforeCursor, cancellationToken);
        else
            return false;
    }
}

Repository

I feel like I've got too many methods here:

public interface IDroidRepository
{
    Task<List<Droid>> GetDroids(
        int? first,
        DateTime? createdAfter,
        CancellationToken cancellationToken);
            
    Task<List<Droid>> GetDroidsReverse(
        int? first,
        DateTime? createdAfter,
        CancellationToken cancellationToken);
            
    Task<bool> GetHasNextPage(
        int? first,
        DateTime? createdAfter,
        CancellationToken cancellationToken);
            
    Task<bool> GetHasPreviousPage(
        int? last,
        DateTime? createdBefore,
        CancellationToken cancellationToken);
            
    Task<int> GetTotalCount(CancellationToken cancellationToken);
}

Cursors

I created a Cursor helper class to help turn any property of any arbitrary type into an opaque base64 string cursor. The code looks like this:

public static class Cursor
{
    private const string Prefix = "arrayconnection";

    public static T FromCursor<T>(string cursor)
    {
        if (string.IsNullOrEmpty(cursor))
            return default;

        string decodedValue;
        try
        {
            decodedValue = Base64Decode(cursor);
        }
        catch (FormatException)
        {
            return default;
        }

        var prefixIndex = Prefix.Length + 1;
        if (decodedValue.Length <= prefixIndex)
            return default;

        var value = decodedValue.Substring(prefixIndex);
        return (T)Convert.ChangeType(value, typeof(T), CultureInfo.InvariantCulture);
    }

    public static (string firstCursor, string lastCursor) GetFirstAndLastCursor<TItem, TCursor>(
        IEnumerable<TItem> enumerable,
        Func<TItem, TCursor> getCursorProperty)
    {
        if (getCursorProperty == null)
            throw new ArgumentNullException(nameof(getCursorProperty));
        if (enumerable == null || enumerable.Count() == 0)
            return (null, null);

        var firstCursor = ToCursor(getCursorProperty(enumerable.First()));
        var lastCursor = ToCursor(getCursorProperty(enumerable.Last()));
        return (firstCursor, lastCursor);
    }

    public static string ToCursor<T>(T value)
    {
        if (value == null)
            throw new ArgumentNullException(nameof(value));

        return Base64Encode(string.Format(CultureInfo.InvariantCulture, "{0}:{1}", Prefix, value));
    }

    private static string Base64Decode(string value) => Encoding.UTF8.GetString(Convert.FromBase64String(value));

    private static string Base64Encode(string value) => Convert.ToBase64String(Encoding.UTF8.GetBytes(value));
}

I initially raised this as a PR at https://github.com/graphql-dotnet/graphql-dotnet/pull/678 but got confused by @jquense. I'm hoping to pickup that coversation here.

Ideas

One idea I have is if creating a connection was as simple as implementing one of two interfaces that might looks something like this:

this.Connection<DroidObject>()
    .Name("droids")
    .Description("Gets pages of droids.")
    .PageSize(MaxPageSize)
    .ResolveAsync<IBidirectionalConnectionResolver<Droid, DateTime?>>();

public interface IBidirectionalConnectionResolver<TModelType, TPropertyType>
{
    // The property on the model we want to use for the cursor.
    TPropertyType GetProperty(Func<TModelType> model);
    
    GetItems(int? first, TPropertyType? after, CancellationToken cancellationToken);
    
    GetItemsReverse(int? last, TPropertyType? before, CancellationToken cancellationToken);
    
    HasNextPage(int? first, TPropertyType? after, CancellationToken cancellationToken);
    
    HasPreviousPage(int? last, TPropertyType? before, CancellationToken cancellationToken);
    
    GetTotalCount(CancellationToken cancellationToken);
}
    
public interface IConnectionResolver<TModelType, TPropertyType>
{
    // The same but for a single direction instead of bi-directional.
}

[benmccallum]

This is definitely a fun one. I may have some ideas as I'm currently implementing this in my own project. Will share my experience when I have more. It's super tricky building these out from what I've done so far.

[RehanSaeed]

Agreed. It gets even trickier if you add a DataLoader. I've had this sample bookmarked for a while which shows how to implement it. Could definately be a lot simpler if we remove all the boilerplate.

[benmccallum]

That sample looks fairly basic in that it's not calculating hasNext or hasPrev..., just setting true. But it's good to get a feel for the structure of that approach.

I think I've got my code to a point where it works on a bidirectional example with dataloader in the mix. I'm basically building up a getItems queryable expressions for the hasNextPage and hasPreviousPage so that I can just pass that as an Any on the end of the getItems queryable.

That's hopefully goin gto allow me to make it so I can do paging by passing a getCursorExpr that is basically an expression for grabbing (and potentially concating) one or many properties on the entity. Problem I'm suspecting will be that EF won't know how to unravel the expression into a query, at which point I'll look at Computed properties and having a Cursor property on every entity.

I don't see it happening yet, but I'm definitely foreseeing having a "sortBy" enum coming into a query that would need to toggle to use a different cursor, so there could even be multiple cursor properties on an entity that I'd need to toggle between accordingly.

When I get it all working, I'll chuck up some source in a gist or something. There'll be code in it that doesn't apply specific to my implementation, but the general concept would be sound.

[benmccallum]

Linking this here as it'll definitely pave the way for EF-based solutions once complete. https://github.com/aspnet/EntityFrameworkCore/issues/14104

[corstian]

@benmccallum The connection and data loader sample wasn't really meant to be anything more than a sample on how to wire it up. In the meantime I finished a long blogpost detailing how I implement connections myself (end to end). Find the post here.

On a proper implementation of hasNextPage and hasPreviousPage, take a look at this part. A bit more context from a previous post might be helpful.

On the topic of ordering results, take a look at this section.

Is this of any help?

[benmccallum]

Thanks @corstian, I really like your solution and blog post; I think it's the best option around at the moment until we sort out the EF Core solution!

[HorjeaCosmin]

Hi @corstian,

Now, when EF.Core has finished to implement SkipWhile and TakeWhile methods, do you think about to write a post using EF implementation instead of using SqlKata?

Tks

[corstian]

@HorjeaCosmin I haven't really seen https://github.com/dotnet/efcore/issues/17065 moving forward yet, though there has been some interesting work going on in https://github.com/dotnet/efcore/issues/20967, which I might link to in a blog post later on. This still isn't an EF Core solution, or am I missing something?

[HorjeaCosmin]

You are right, sorry, I was confused by SkipWhile Linq.Queryable extension.