Flaky Tests

[darrellwarde]

Causal cluster issues

These tests frequently fail because we possibly insert data using Cypher, but don't pass the resulting bookmark into the GraphQL query. This means that the cluster member that our query lands on may not be aware of test data.

• [ ] FAIL packages/graphql/tests/integration/issues/1364.int.test.ts
  • https://github.com/neo4j/graphql/issues/1364 › Should project cypher fields after applying the sort when sorting on a non-cypher field on a root connection)
• [ ] FAIL packages/graphql/tests/integration/issues/988.int.test.ts
  • https://github.com/neo4j/graphql/issues/988 › should query nested connection
• [ ] FAIL packages/graphql/tests/integration/issues/560.int.test.ts ● https://github.com/neo4j/graphql/issues/560 › should not throw when CartesianPoint is null
• [ ] FAIL packages/graphql/tests/integration/issues/1782.int.test.ts (5.861 s) ● https://github.com/neo4j/graphql/issues/1782 › should only return the single 'chain' and the multiple nested 'chains', three relations deep, using SOME

Arrays not being checked in unordered way

Unless we are confident about the order of an array, we need to check it using a combination of expect.toIncludeSameMembers() from jest-extended.

• [x] FAIL packages/graphql/tests/integration/issues/1221.int.test.ts ● https://github.com/neo4j/graphql/issues/1221 › should return as two single 'chains', three relations deep ● still flaky 26.09.2022

Tests with non-unique type names

There are tests that don't use unique type names which causes test linkage.

• [x] FAIL packages/graphql/tests/integration/directives/auth/is-authenticated.int.test.ts

Subscriptions E2E

• [ ] FAIL packages/graphql/tests/e2e/subscriptions/delete-relationship-via-delete-with-relationships.e2e.test.ts (51.251 s) ● Delete Subscriptions when relationships are targeted- with interfaces, unions and nested operations › disconnect via delete nested - with relationships: interface to union type

  expect(received).toHaveLength(expected)
  
  Expected length: 16
  Received length: 10
   5498 |         expect(wsClient2.errors).toEqual([]);
   5499 |
  > 5500 |         expect(wsClient2.events).toHaveLength(16);
        |                                  ^
   5501 |         expect(wsClient.events).toHaveLength(5);
   5502 |
   5503 |         expect(wsClient2.events).toIncludeSameMembers([
  
   at Object.<anonymous> (packages/graphql/tests/e2e/subscriptions/delete-relationship-via-delete-with-relationships.e2e.test.ts:5500:34)
  

Unknown issues

• [ ] FAIL packages/graphql/tests/integration/root-connections.int.test.ts
  • root-connections > should return an array of edges and the correct totalCount
• [ ] FAIL packages/graphql/tests/integration/aggregations/top-level/float.int.test.ts ● aggregations-top_level-float › should return the sum of node properties Expected 10.4, received 10.33333339

[neo4j-team-graphql]

Many thanks for raising this bug report @darrellwarde. :bug: We will now attempt to reproduce the bug based on the steps you have provided.

Please ensure that you've provided the necessary information for a minimal reproduction, including but not limited to:

• Type definitions
• Resolvers
• Query and/or Mutation (or multiple) needed to reproduce

If you have a support agreement with Neo4j, please link this GitHub issue to a new or existing Zendesk ticket.

Thanks again! :pray:

[angrykoala]

After a whole revamp of our integration tests this issue is outdated. Some flakyness still remains, but can track these in separate issues as we find them