Explore possibility of generic types

[AndrewIngram]

As projects like Relay have shown, it's relatively common to repeat the same generic structures of types multiple times within a project. In the case of Relay, I'm talking about Connections.

The GraphQL definition language already has explicit support for one particular form of generic type, arrays:

type Foo {
   id: ID!
   bars: [Bar]
}

I'd like to start discussion about being able to do something similar for user-defined structures:

generic ConnectionEdge<T> {
   node: T
   cursor: String
}

generic Connection<T> {
   edges: ConnectionEdge<T>
   pageInfo: PageInfo
}

type Foo {
   id: ID!
   bars: Connection<Bar>
}

The overall goal is to reduce the amount of boilerplate in creating schemas with repetitive structures.

[leebyron]

Thanks for proposing this! During the GraphQL redesign last year, we actually considered adding this!

There are two issues that seemed to result in more complication than would be worth the value we would get from this feature, but I'm curious what you think or if you have ideas:

1. What does the introspection result look like? When I read the ConnectionEdge type and look at its fields, what will the type of node be? The best answer we could come up with was to change Field.type from Type to Type | GenericParameter which is a bit of a bummer as it makes working with the introspection API more complicated. We could also expand Type to include the possibility of defining a generic param itself. Either way, it also has some rippling effects on the difficulty of implementing GraphQL, which would need to track type parameter context throughout most of it's operations.
2. What should __typename respond with? What should { bars { __typename } } return? This one is pretty tricky. { "bars": { "__typename": "Connection" } }? That describes the type, but you're missing info about the type parameter, that that ok? { "bars": { "__typename": "Connection<Bar>" } } Is also problematic as now to use the __typename field you need to be able to parse it. That also adds some overhead if you were hoping to use it as a lookup key in a list of all the types you know about.

Not to say these problems doom this proposal, but they're pretty challenging.

Another thing we considered is how common type generics would actually be in most GraphQL schema. We struggled to come up with more than just Connections. It seemed like over-generalization to add all this additional complexity into GraphQL just to make writing Connections slightly nicer. I think if there were many other compelling examples that it could motivate revisiting.

[AndrewIngram]

You're right about the number of use cases being relatively small, i'll need to think on that point.

To be honest, this feels like sugar for developers of schemas rather than clients. In the simplest case, i'd just expect the introspection result to be the same as it is now, i.e the generics get de-sugared. To that end, it could just be something that parsers of the schema definition language end up supporting, but it's up to library authors how to handle the generated AST.

In graphql-js land, there are numerous examples of libraries (apollo-server, my own graphql-helpers, and a few others I can't remember) which use the parser provided to vastly simplify the process of building schemas (having done it both ways, I'd say it's pretty close to an order of magnitude more productive), and i'd personally be happy to add additional support for tokens related to generics to my library.

However, it does feel weird supporting a syntax that's not actually reflected in the final generated schema, so i'm unsure about this approach.

[Qard]

I really wish something like this would be reconsidered. Connections may just be a single use-case, but it's a big one, in my opinion. The length of my current schema would cut in half with generics.

Currently I have 24 copies of basically this:

type TypeXConnectionEdge {
   node: TypeX
   cursor: String
}
type TypeXConnection {
   edges: TypeXConnectionEdge
   pageInfo: PageInfo
}

That's nearly 200 lines of code that could easily be expressed in 8 lines of generics. I'm seriously considering writing my own preprocessor just to hack on my own generics capability...

[stubailo]

Hmm, in graphql-tools you could do something like:

type MyType {
  hello: String
  world: String
}

${ connectionAndEdgeFor('MyType') }

Is there something the syntax could have that would be better than that JS-style approach?

[Qard]

And what if you're not using JS? 😞

I want my schema to be pure graphql schema language so it doesn't need preprocessing.

[stubailo]

Yeah I definitely sympathize. I guess the real question is, is the generic thing just something for the server to be written more conveniently, or does the client somehow know that these types/fields are generic and acts accordingly?

If the client doesn't know, then I feel like it should be a preprocessor feature or a macro thing. The spec is all about the contract between client and server IMO.

However, there are definitely implementations for generics where the client could actually take advantage of knowledge that something is a generic thing. For example, in the connection example, there's no way to make an interface that says that a TypeXConnectionEdge should have a node of type X, so you can't really enforce that without relying on conventions.

Perhaps this could be done as some sort of intersection of interfaces and type modifiers? So basically, it's a way of creating your own type modifiers - if you squint hard enough, [ ... ] and ! are kind of like List<T> and NonNull<T>.

So building on that, in introspection:

fragment TypeRef on __Type {
  kind
  name
  ofType {
    kind
    name
  }
}

You could get:

{
  kind: "GENERIC",
  name: "Connection",
  ofType: {
    kind: "OBJECT",
    name: "Photo"
  }
}

Perhaps this should come with a change of kind: "LIST" to kind: "GENERIC", name: "LIST".

[Qard]

I mean, it seems hugely valuable for the client to understand generic concepts too, but that could probably be expressed as simply differently named types, since the client doesn't generally need to be too concerned about the actual name of types so much as what is in them. It seems to me like it'd be really valuable to be able to, in a type-safe way, express concepts like pagination while retaining the safety of recognizing that a given type encapsulates objects of a specific type. Without generics or preprocessing you can sorta-kinda do this with unions, but then you're throwing away the promise that all items in a list are of a specific type...

[stubailo]

I guess in my mind, whether or not the client should worry about it is a very important factor in determining whether it should be in the spec or simply live as some server-side library tooling.

[mike-marcacci]

Hi everybody! I've definitely been looking to solve the Connection use-case here, and another similar case specific to my project. I have a slightly different strategy which I've laid out in #295, which is a much smaller change, and in no way mutually exclusive with this proposal.

Basically, if an interface were able to explicitly implement another interface, the client would be aware of the hierarchy. That is, it would be provided similar context to what generics might provide, but without adding new semantics to the protocol.

This wouldn't solve the issue of verbosity within the schema, but leverage the existing types to convey "generic" type information to the client. In this way, a preprocessor might be able to compile generics into hierarchal interfaces, achieving both goals here.

[AndrewIngram]

Given that we now have at least one authoring-only syntax extension, i.e. extend type, is it worth reconsidering generics in the same light?

[stubailo]

Hmmm, that's true - extend type is not accessible through introspection at all, I didn't think about that.

[AlecAivazis]

One use case I have run into for generics is having something resembling Maybe<T> to handle error messages. I'd like to do so without having to mess with the network layer and introduce some sort of global state, or refer to a disjointed part of the response. Currently, I am defining a separate union type for each object (ie MaybeUser is a User | Error) but it would be nice to be able to do this as simply as Maybe<User> and define the structure once.

An alternative to avoid the extra complexity of the generic union would be something as simple as

generic Maybe<T> {
   left: Error
   right: T
}

[xialvjun]

Another use case is

generic Pagination<T> {
    total: Int!
    limit: Int!
    offset: Int!
    results: [T!]!
}

type Person {
    id
    name
    # ...
}

type Query {
    search_person(limit: Int = 20, offset: Int = 0, q: String): Pagination<Person>!
}

[crypticmind]

generic Maybe<T> { ... } and generic Pagination<T> { ... } look great to me, though I'd drop the generic keyword as it seems redundant by the use of angle brackets.

[iamdanthedev]

Addressing problems @leebyron mentioned in the 2nd post here, generics don't need to appear in introspection at all as they are abstract helpers. Concrete types extending generics could be introspected with all the precision of graphql

generic Pagination<T> {
    total: Int!
    limit: Int!
    offset: Int!
    results: [T!]!
}

type Person {
    id
    name
    # ...
}

type PaginatedPersons extends Pagination<Person> {
  extraField: Int! // maybe
}

type Query {
    search_person(limit: Int = 20, offset: Int = 0, q: String): PaginationPersons
}

In this case the type PaginatedPersons could have in introspection this shape

type PaginatedPersons {
    total: Int!
    limit: Int!
    offset: Int!
    results: [Person!]!
    extraField: Int!
}

[dallonf]

I think there is a potential use case for client-side generics in enabling re-usable components. Example, using react-apollo, some hypothetical syntax to inject a fragment into another fragment, and the Pagination<T> type from the above post:

const PaginatedListFragment = gql`
# some purely hypothetical syntax on how you might inject
# another fragment into this one
fragment PaginatedList($itemFragment on T) on Pagination<T> {
  total
  offset
  limit
  results {
    id
    ...$itemFragment
  }
}
`;

const PaginatedList = ({ data, renderItem, onChangeOffset }) => (
  <div>
    {data.results.map(item => <div key={item.id}>{renderItem(item)}</div>)}
    <PaginationControls
      total={data.total}
      offset={data.offset}
      limit={data.limit}
      onChangeOffset={onChangeOffset}
    />
  </div>
);

const PeoplePageQuery = gql`
query PeoplePage($limit: Int, $offset: Int) {
  people(limit: $limit, offset: $offset) {
    # again, hypothetical parametric fragment syntax
    ...PaginatedList($itemFragment: PersonItemFragment)
  }
}
${PaginatedListFragment}

fragment PersonItemFragment on Person {
  id
  name
}
`;

const PeoplePage = ({ data }) => (
  <div>
    <h1>People</h1>
    <PaginatedList
      data={data.people}
      renderItem={item => <a href={`/person/${item.id}`}>{item.name}</a>}
      onChangeOffset={offset => data.setVariables({ offset })}
    />
  </div>
);
]

This would be something very powerful for component-based frameworks!

[AndrewIngram]

@dallonf this was actually one of the other benefits I envisaged :)

It's the primary reason why I don't think it's enough for this to just be syntactic sugar for the existing capabilities, allowing clients to know about generics could be a very powerful feature.

[kbrandwijk]

I also have another use case for generics. I want to define an array of distinct elements, commonly knows as a Set. So instead of defining a field like myValues: [String], I would like to define it as myValues: Set<String> to accomplish this.

[mike-marcacci]

To add another use case, I've found myself making custom scalars when I would otherwise define a Map<SomeType>. In these cases SomeType has always been a scalar so this has been acceptable, but there have been other times when I chose to use [SomeObjectType] when a real Map<SomeObjectType> would have been preferable.

-- edit --

I do want to note, though, that this would be quite a challenge to implement, since the generic's definition in the schema would need to fully describe the final structure. Otherwise, there's no way for a client to know what to expect.

[axos88]

Another use, also tied with pagination is when different interfaces need to be paginated.

Suppose we have an interface Profile, and two implementations: UserProfile and TeamProfile

Without generics, I cannot see a solution to be able to deduct that UserProfilePagnated and TeamProfilePaginated can be supplied where we are expecting PaginedProfiles. With generics - although not trivial, one can make a complex analyzer that understands the notion of co and contravariance, and can deduct that PaginatedProfiles is an interface of UserPaginatedProfiles, although one might need to signal this somehow.

[mike-marcacci]

Hi @axos88, can you add your comment on #295? Your use case is exactly the kind of problem it is designed to address.

[axos88]

@mike-marcacci I'm not sure how interfaces implementing interfaces is enough to have a solution for the problem

[axos88]

Although I agree, the problem is related.

[axos88]

Oh I see. :)

[akomm]

@leebyron

What if generic types can not be returned by fields without type specification. They are just for boilerplate reduction. As you can see in the following definition (from @AndrewIngram's post) the field bars does not return the generic type over T, but Bar:

generic ConnectionEdge<T> {
   node: T
   cursor: String
}

generic Connection<T> {
   edges: ConnectionEdge<T>
   pageInfo: PageInfo
}

type Foo {
   id: ID!
   bars: Connection<Bar>
}

And the generic types are not visible in introspection.

Result of this could be:

type ConnectionEdgeBar {
   node: Bar
   cursor: String
}

type ConnectionBar {
   edges: ConnectionEdgeBar
   pageInfo: PageInfo
}

type Foo {
   id: ID!
   bars: ConnectionBar
}

As far as I know type name duplication is not allowed in graphql schema and all implementation raise errors in case of duplication. So when you already have ConnectionBar somewhere, then you would get error.

You do not have to drop the information when schema is parsed, so you could point out that ConnectionBar being generated from generic Connection<T> is already defined. In introspection the schema is already processed and the there only exists the ConnectionBar.

I can't speak for others, but I have a lot of boilerplate, which would be saved if I could use generics like that. I have noticed, that on field-level (except generic type fields) I do not need to be generic in this case. I can defined specific type there. No need to return generic<T,V> etc.

[fbartho]

@akomm's recommendation is the style that I've gone with in my project.

I currently implemented it with @generic(...) as a directive, that I preprocess out to generate the types:

interface Page @generic(over: [DataType]) {
	items: [DataType]
	meta: PageMeta
}

type PageMeta {
	page: Int
	totalPages: Int
	totalItems: Int
}

# Elsewhere:

type PopularItems_ implements Page
	@generic(using: [PopularItem_]) {
	searchSectionTitle: String!
}

type PopularItem_ {
	id: ID!
	title: String!
	iconUrl: String
	narrowPhotoUrl: String!
	slug: String!
}

After pre-processing, the fields from the Page interface are merged in to the fields of PopularTemplates_ after distributing the using-parameter over all instances of DataType

[akomm]

So basically graphql need a way to define opaque types/templates to reduce boilerplate, which are not directly exposed as API. For generic results we have interfaces.

Those templates could have an optional annotation to define how the final type's name is composed - or if it makes sense, a fixed method how it is composed.

[AndrewIngram]

As I mentioned in a previous comment, there are reasons why we might not just want this as definition-time syntactic sugar. Exposing the concept of generics to API consumers would allow for highly-reusable client-side code that exploits them. It's currently relatively verbose to implement different connection pagination containers with Relay even though the structure is almost identical every time. Generics are an incredibly powerful construct, so it'd be nice to be able to enhance the GraphQL type system with them.

The main argument against them seems to be that it would increase the difficulty of implementing GraphQL libraries (eg graphql-js, sangria, graphene, absinthe). If the client-side value is limited, i'd agree that it's not worth the complexity. But i'd argue that it's not worth adding at all if it's just a syntactic sugar thing -- there's nothing stopping people implementing these patterns (if slightly less elegantly) in user-land today.

[alamothe]

Generics could also be useful to model type-safe IDs:

type Person {
  id: ID<Person>!,
  name: String!
}

type Query {
  person(id: ID<Person>!): Person!
}

[akomm]

@alamothe

Depends on how you define ID. If it is relay-like opaque string, what would the generic ID<T> type look like?