Pagination model

[MichalLytek]

I think it might be nice to have a pagination model that will allow to create universal resolvers, types definitions, etc. for collections.

For now I am going to expose two types of models:

Simple pagination

Like with standard rest apis - only offset (skip), limit (take), count and items.

@Resolver()
class SampleResolver {
  @Query(returns => [Type])
  paginatedQuery(
    @PaginationArgs() { offset, limit }: PaginationArgs,
  ): PaginationResponse<Type> {
    // ...
    return [data, count];
  }
}

By using @PaginationArgs() it will detect that the query is being paginated, so the returned tuple [TData, number] will be converted into an object with page info (hasNextPage, etc.), data and total properties.

Relay connection

Using cursor model, which is better for highly dynamic collections: https://facebook.github.io/relay/graphql/connections.htm

API - TBD.

[erencay]

@19majkel94 Sorry if this is mostly irrelevant but recently I started using dedicated query and mutation resolvers which extends abstract resolver classes with an abstract resolve method. Under the hood they can hide a lot of boilerplate and provides a clean implementation. Each class represents one and only one operation with a naming strategy based on the class name. I have yet to implement this solution for pagination but it looks like this approach works the best for me, so far.

The thing is, this is something resembles may be what a "conventional framework" would do? I don't know if this kind of approach is beyond type-graphql's intention. Also typegraphql is successfully using the decorator pattern which you can simply plug-in/out features. Inheritance might not have it's place in it to solve such problems in the first place.

Just wanted to point out how I'm planning to solve the problem.

@InputType()
class LoginInput {
    @Field()
    public username: string;

    @Field()
    public password: string;
}

@MutationResolver(LoginInput, type => Login)
class Login extends AbstractMutationResolver<LoginInput> {
    public async resolve() {
        if (this.ctx.currentUser) {
            throw new Error("You are already logged in.")
        }

        return this.input.username == this.input.password;
    }
}

[MichalLytek]

I'm glad that you found a way to reduce a boilerplate for your case 😃 I would love to take a look at it when you finish it.

But I think that such constructs that you've presented is way too much complicated design to be the official API. Of course, it can reduce boilerplate but also introduces a huge amount of magic which is not intuitive.

TypeGraphQL design goal is to be simple yet powerful. The decorators to schema idea is understandable, features like authorization or validation are powerful yet simple. So do the pagination API has to be simple and easy to use, like the one in https://github.com/indigotech/graphql-schema-decorator.

[tonyxiao]

@19majkel94 what's the recommended way to do pagination until there's official support for pagination in type-graphql?

[tonyxiao]

Just tested - this seems to work for defining the graphql data models. Now going to experiment with how to actually use this efficiently with something like typeorm

import * as Relay from 'graphql-relay'
import { ArgsType, ClassType, Field, ObjectType } from 'type-graphql'

export type ConnectionCursor = Relay.ConnectionCursor

@ObjectType()
export class PageInfo implements Relay.PageInfo {
  @Field()
  hasNextPage!: boolean
  @Field()
  hasPreviousPage!: boolean
  @Field()
  startCursor?: ConnectionCursor
  @Field()
  endCursor?: ConnectionCursor
}

@ArgsType()
export class ConnectionArgs implements Relay.ConnectionArguments {
  @Field({ nullable: true, description: 'Paginate before opaque cursor' })
  before?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate after opaque cursor' })
  after?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate first' })
  first?: number
  @Field({ nullable: true, description: 'Paginate last' })
  last?: number
}

export function connectionTypes<T extends ClassType>(name: string, nodeType: T) {
  @ObjectType(`${name}Edge`)
  class Edge implements Relay.Edge<T> {
    @Field(() => nodeType)
    node!: T

    @Field({ description: 'Used in `before` and `after` args' })
    cursor!: ConnectionCursor
  }

  @ObjectType(`${name}Connection`)
  class Connection implements Relay.Connection<T> {
    @Field()
    pageInfo!: PageInfo

    @Field(() => [Edge])
    edges!: Edge[]
  }
  return {
    Connection,
    Edge,
  }
}

export {
  connectionFromArray,
  connectionFromPromisedArray,
  connectionFromArraySlice,
  connectionFromPromisedArraySlice,
} from 'graphql-relay'

[MichalLytek]

what's the recommended way to do pagination until there's official support for pagination in type-graphql?

Manually 😆

Just tested - this seems to work for defining the graphql data models.

That's right but without #180 you can't easily add some properties to edge or connection types.

I wonder if I should couple cursor pagination with graphql-relay making it a dependency or create my own helpers for returning paginated results 😕

[tonyxiao]

the implementation of pagination feels database dependent. I used this lib https://github.com/darthtrevino/relay-cursor-paging/ to interpret the ConnectionArgs into limit and offset, then used graphql-relay to turn result set into the correct form. non sql database may do it differently tho. that said no matter which one you choose I assume graphql-relay helpers can be applicable.

[cipriantarta]

Just tested - this seems to work for defining the graphql data models. Now going to experiment with how to actually use this efficiently with something like typeorm

How would I go about using that? 😬

[tonyxiao]

@cipriantarta connectionPaging.ts

// Based on https://github.com/darthtrevino/relay-cursor-paging
import { ArgsType, Field } from 'type-graphql'

/**
 * TODO: Figure out how to validate this with class-validator
 * https://github.com/typestack/class-validator/issues/269
 */
@ArgsType()
export class ConnectionArgs implements ConnectionArguments {
  @Field({ nullable: true, description: 'Paginate before opaque cursor' })
  before?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate after opaque cursor' })
  after?: ConnectionCursor
  @Field({ nullable: true, description: 'Paginate first' })
  first?: number
  @Field({ nullable: true, description: 'Paginate last' })
  last?: number

  pagingParams() {
    return getPagingParameters(this)
  }
}

type PagingMeta =
  | { pagingType: 'forward'; after?: string; first: number }
  | { pagingType: 'backward'; before?: string; last: number }
  | { pagingType: 'none' }

function checkPagingSanity(args: ConnectionArgs): PagingMeta {
  const { first = 0, last = 0, after, before } = args
  const isForwardPaging = !!first || !!after
  const isBackwardPaging = !!last || !!before

  if (isForwardPaging && isBackwardPaging) {
    throw new Error('cursor-based pagination cannot be forwards AND backwards')
  }
  if ((isForwardPaging && before) || (isBackwardPaging && after)) {
    throw new Error('paging must use either first/after or last/before')
  }
  if ((isForwardPaging && first < 0) || (isBackwardPaging && last < 0)) {
    throw new Error('paging limit must be positive')
  }
  // This is a weird corner case. We'd have to invert the ordering of query to get the last few items then re-invert it when emitting the results.
  // We'll just ignore it for now.
  if (last && !before) {
    throw new Error("when paging backwards, a 'before' argument is required")
  }
  return isForwardPaging
    ? { pagingType: 'forward', after, first }
    : isBackwardPaging
      ? { pagingType: 'backward', before, last }
      : { pagingType: 'none' }
}

const getId = (cursor: ConnectionCursor) => parseInt(fromGlobalId(cursor).id, 10)
const nextId = (cursor: ConnectionCursor) => getId(cursor) + 1

/**
 * Create a 'paging parameters' object with 'limit' and 'offset' fields based on the incoming
 * cursor-paging arguments.
 *
 * TODO: Handle the case when a user uses 'last' alone.
 */
function getPagingParameters(args: ConnectionArgs) {
  const meta = checkPagingSanity(args)

  switch (meta.pagingType) {
    case 'forward': {
      return {
        limit: meta.first,
        offset: meta.after ? nextId(meta.after) : 0,
      }
    }
    case 'backward': {
      const { last, before } = meta
      let limit = last
      let offset = getId(before!) - last

      // Check to see if our before-page is underflowing past the 0th item
      if (offset < 0) {
        // Adjust the limit with the underflow value
        limit = Math.max(last + offset, 0)
        offset = 0
      }

      return { offset, limit }
    }
    default:
      return {}
  }
}

then in a custom repo.

  async findAndPaginate(conditions: FindConditions<T>, connArgs: ConnectionArgs) {
    const { limit, offset } = connArgs.pagingParams()
    const [entities, count] = await this.findAndCount({
      where: conditions,
      skip: offset,
      take: limit,
    })
    const res = connectionFromArraySlice(entities, connArgs, { arrayLength: count, sliceStart: offset || 0 })
    return extendPageInfo(res, {
      count,
      limit,
      offset,
    })
  }

[cipriantarta]

@tonyxiao Thanks for the details, I was mostly concerned about calling your method connectionTypes, but I think the signature should be export function connectionTypes<T extends ClassType>(name: string, nodeType: ClassType<T>), otherwise I get complains when trying to populate the edges.

[tonyxiao]

Actually, this is my latest version of that

import { TypeValue } from 'type-graphql/decorators/types'
export function connectionTypes<T extends TypeValue>(name: string, nodeType: T) {
  @ObjectType(`${name}Edge`)
  class Edge implements Relay.Edge<T> {
    @Field(() => nodeType)
    node!: T

    @Field({ description: 'Used in `before` and `after` args' })
    cursor!: ConnectionCursor

    @Field(() => GQLJSON)
    cursorDecoded() {
      return Relay.fromGlobalId(this.cursor)
    }
  }

  @ObjectType(`${name}Connection`)
  class Connection implements Relay.Connection<T> {
    @Field()
    pageInfo!: PageInfo

    @Field(() => [Edge])
    edges!: Edge[]
  }
  return {
    Connection,
    Edge,
  }
}

[cipriantarta]

Looks great. Thanks again Tony!

On Thu, 25 Oct 2018 at 19:58, Tony Xiao [email protected] wrote:

Actually, this is my latest version of that

import { TypeValue } from 'type-graphql/decorators/types'export function connectionTypes<T extends TypeValue>(name: string, nodeType: T) { @ObjectType(${name}Edge) class Edge implements Relay.Edge<T> { @Field(() => nodeType) node!: T

@Field({ description: 'Used in `before` and `after` args' })
cursor!: ConnectionCursor

@Field(() => GQLJSON)
cursorDecoded() {
  return Relay.fromGlobalId(this.cursor)
}

}

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/19majkel94/type-graphql/issues/142#issuecomment-433128240, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2skeM8TYtcrIyE53HDrAnE05Y6XOVOks5uoe23gaJpZM4WWrcs .

[Veetaha]

@19majkel94 It would be great if we could get Class/FieldMetadata from classes decorated with @ObjectType, @Field and dynamically generate other @Object/InputType with statically unknown properties, so that implementing generic pagination with filtering higher order resolver would be possible. But now we can't inspect defined property names and types from the given @ObjectType class in generic context, which (I think) is not hard to expose. But there should be another way of defining @Object/InputType except via decorators. Could you please consider?

[MichalLytek]

@Veetaha TypeGraphQL design goal is not to dynamically create derived types in runtime that can't be described in compile time (which means using any as an arg type).

I have in plans an API that would allow to transform types like the TS Partial, Pick, etc. For more advanced use cases, you may use a code generator that will parse the base model class and produce new derived types/classes.

[Veetaha]

@19majkel94 I understand that this was not your goal, though do you think this would be excessive? I am not a big fan of code-generation, because it only emphasizes that the given tool failed to reduce boilerplate. I don't see any obstacles for allowing dynamic creation of resolvers/types, as GrahQL has no generics and very restricted build-in tools to achieve it, so maintaining such a feature would lead to much less boilerplate. Please, don't refuse)

And why do you say

(which means using any as an arg type)

I try to avoid any type and use unknown if needed, moreover, when I stumble with any I just throw in some generics and get the strongest typing.

Once agian about dynamic resolvers/types. I suppose we can "add decorators dynamically" by manually invoking them with the proper target arguments e.g. Field(_type => Boolean)(ObjClass.prototype, 'propName'). But still, a disability to get ClassMetadata from the given constructor function limits metaprogramming (( Can you guarantee this feature in future or at least consider it?

[MichalLytek]

No. The goal is to have 1:1 mapping of TS types to GraphQL types. I will try to provide as much convenient tools for that, but I won't create a dynamic API for generating GraphQL types without TS types - you can use graphql-js for that. If you like that, I can provide an interop with it by passing type => GraphQLObjectType in decorator but it's a dirty and not recommended workaround.

[Qix-]

Having generators would be useful in certain cases. If I'm returning a somewhat large data set, I don't want to have to enumerate it all in memory before sending it to the client, but rather stream it. Generators would be wonderful here. I'm not sure if that's possible with Apollo, however.

[MichalLytek]

If you mean function * by generators, it's not possible, even by GraphQL spec (streaming chunks of data).

For large datasets, it may be better to just use pagination for manual paging.

[tonyxiao]

There is @defer directive though. Not quite pagination but maybe potentially relevant?

[Veetaha]

@19majkel94 Interesting to peek the definition of PaginationResponse<Type> as graphql doesn't support tuples, by the way, this a little bit annoying having to map typeorm's pagination tuple to { data: T[]; total: number; } object each time, but this is out of scope.

[subvertallchris]

@tonyxiao Any chance you have a full example of your approach in a gist or repo anywhere?

[ceefour]

@tonyxiao where did you import PageInfo from?

[tonyxiao]

@ceefour it's not imported, but part of the snippeet above. @subvertallchris we actually aren't using GraphQL anymore at the moment, so don't have easy access to a full example. :(

[Superd22]

@ceefour just fyi we've been using https://github.com/wemaintain/auto-relay in prod to great avail and are actively maintaining it.

[ceefour]

thanks @Superd22 , I've been writing the PageInfo etc. classes myself. not too bad actually although pretty repetitive. Your library seems very interesting and useful!

Definitely what I was looking for. (and what I expected at least core of it is in typegraphql)

[MichalLytek]

and what I expected at least core of it is in typegraphql

@ceefour I'm a sole developer here with a full-time job. I would like to have everything in TypeGraphQL but I have to choose and give priorities to features 😞

[Superd22]

I 100% agree that this kind of "side" and somewhat opinionated (not everyone uses relay) features cannot be the priority when there's only one core contributor to an open source project.

The decision to create a separate package rather than submit a PR to this repo was made because we needed something working asap and it seemed to be better to wait until typegraphql ships in 1.0 and becomes a monorepo to implement this.

We 100% plan on helping porting some features of auto-relay into typegraphql if and when appropriate.

[MOTORIST]

@ceefour Pagination has different implementations (offset, relay, ...) and not required to use ORM. What implementation should be in the typegraphql?

[ceefour]

@MOTORIST You're right. On second thought I'm already happy that the Relay choice is provided by @wemaintain's project.

Perhaps a better approach is to endorse that project from TypeGraphQL's documentation?

[lemiesz]

I think it would be better to have a TypeGQL-extensions (TypeGQL-plugins) repo. That could support custom utilities like this. While I would definitely use this pagination feature, I think its nicer to keep the core library minimal.

[MichalLytek]

Perhaps a better approach is to endorse that project from TypeGraphQL's documentation?

Sure we can make a section in docs for community libraries/plugins 😉 Feel free to make a PR if you find other extensions to TypeGraphQL.