octokit.graphql.net icon indicating copy to clipboard operation
octokit.graphql.net copied to clipboard

Nested pagination problem

Open samngmco opened this issue 2 years ago • 0 comments

Refer to THIS, I have a query with nested AllPages()

My problem is when querying teams --nested--> members. In my org, we have ~300 teams and one particular team has ~180 members. The problem is I only get the first 200 teams.

I believe the bug is this: when there is pagination happening on the nested objected, the code will handle the child pagination in the next call, but then it will not continue querying the pagination of the parent object

var query = new Query()
    .Organization(Var("owner"))
    .Teams()
    .AllPages()
    .Select(team => new
    {
        team.Name,
        Owner = team.Organization.Login,
        Slug = team.CombinedSlug,
        team.CreatedAt,
        team.UpdatedAt,
        childsTeams = team.ChildTeams(null, null, null, null, Var("immediateOnly"), null, null)
            .AllPages()
            .Select(team2 => new
            {
                Slug = team2.CombinedSlug
            }).ToList(),
        members = team.Members(null, null, null, null, Var("memberType"), null, null, null)
            .AllPages()
            .Select(user => new
            {
                user.Login
            }).ToList()
    }).Compile();

First Request

{
  "query":"query($owner:String!,$immediateOnly:Boolean,$memberType:TeamMembershipType){organization(login:$owner){id teams(first:100){pageInfo{hasNextPage endCursor} nodes{id name owner: organization{login} slug: combinedSlug createdAt updatedAt childsTeams: childTeams(immediateOnly:$immediateOnly,first:100){pageInfo{hasNextPage endCursor} nodes{slug: combinedSlug}} members(membership:$memberType,first:100){pageInfo{hasNextPage endCursor} nodes{login}}}}}}",
  "variables":{"owner":"helloworld","immediateOnly":true,"memberType":"IMMEDIATE"}
}

First Response

{
  "data": {
    "organization": {
      "id": "MDEyOk9yZ2FuaXphdGlvbjI5XXXXXXXX",
      "teams":{
        "pageInfo": { 
          "hasNextPage": true,
          "endCursor": "Y3Vyc29yOnYyOpMCrGRhdGEtc2NpZWXXXXXXXXXX"
      },
      "nodes":[...

Second Request (start with the correct cursor)

{
  "query":"query($__id:ID!,$__after:String,$immediateOnly:Boolean,$memberType:TeamMembershipType){node(id:$__id){__typename ... on Organization{teams(first:100,after:$__after){pageInfo{hasNextPage endCursor} nodes{id name owner: organization{login} slug: combinedSlug createdAt updatedAt childsTeams: childTeams(immediateOnly:$immediateOnly,first:100){pageInfo{hasNextPage endCursor} nodes{slug: combinedSlug}} members(membership:$memberType,first:100){pageInfo{hasNextPage endCursor} nodes{login}}}}}}}",
  "variables":{"owner":"helloworld","immediateOnly":true,"memberType":"IMMEDIATE","__id":"MDEyOk9yZ2FuaXphdGlvbjI5XXXXXXXX","__after":"Y3Vyc29yOnYyOpMCrGRhdGEtc2NpZWXXXXXXXXXX"}
}

Second Response

In the second response, the hasNextPage is still true, therefore, the code should continue to query teams in the next page. However, one of the nested group (the members node), also has a hasNextPage=true

{
  "data": {
    "node": {
      "__typename": "Organization",
      "teams": {
        "pageInfo":{
          "hasNextPage": true,
          "endCursor": "Y3Vyc29yOnYyOpMCs3FhLXRva2VuLXBXXXXXXXXXXXXXXXXXXX=="
        },
        "nodes":[
        ...
        {
        "id": "MDQ6VGVhbTXXXXXXXXX=",
        "name": "foobar_team",
        "owner":{"login":"helloworld"},
        "slug":"helloworld/foobar_team",
        "createdAt":"2019-10-08T01:02:45Z",
        "updatedAt":"2021-08-03T06:58:44Z",
        "members": {
          "pageInfo": { 
            "hasNextPage": true,
            "endCursor":"Y3Vyc29yOnYyOpXXXXXXXX=="
          },
          "nodes": [
          ...100 items here
          ]

Third Request

The third request is to retrieve the reminding members of the foobar_team, which is correct. But after that, the program is supposed to continue to query the reminding teams, but it didn't happened.

samngmco avatar Jun 07 '22 16:06 samngmco