defer stream support within stitching

[yaacovCR]

A work in progress...

Help wanted...

• [x] test single proxied deferred patch.
• [x] test merging deferred patches from multiple subschemas
• [x] handle stream

In terms of mechanism, have settled on external values that are from asyncIterables being converted to an initial result with an associated Receiver object, which is passed down the execution tree. If a field is not present on the result, and a Receiver can be found, the default merging resolver will request to be notified when the field is available.

• [x] think more broadly about how to implement the Receiver, maybe we should use Repeaters rather than raw async iterables, and what the best pattern is for consuming async iterables.... see https://github.com/repeaterjs/repeater/issues/48#issuecomment-747651900

• [ ] update all transforms that touch results - some result transformers parse the request and transform the response based on saved state -- these transformers may require rewriting patch labels so that state does not overlap between different fragments...

  • [x] CheckResultAndHandleErrors (merging errors into results) - replaced by externalValueFromResult and externalValueFromPatchResult functions
  • [ ] TransformQuery
  • [ ] MapLeafValues
  • [ ] RenameRootTypes
  • [ ] RenameTypes
  • [ ] TransformCompositeFields
  • [ ] WrapFields
  • [ ] HoistFields

• [x] update batch-execute

• [ ] update batch-delegate

• [ ] test errors

• [ ] test timeouts

https://github.com/graphql/graphql-js/pull/2319 https://github.com/graphql/graphql-spec/pull/742

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: 3a079fc30c01ee3984e7715eb5a4937927842d31

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 25 packages

Name	Type
@graphql-tools/delegate	Major
@graphql-tools/mock	Major
@graphql-tools/utils	Major
@graphql-tools/wrap	Major
@graphql-tools/batch-delegate	Major
@graphql-tools/batch-execute	Major
graphql-tools	Major
@graphql-tools/stitch	Major
@graphql-tools/links	Patch
@graphql-tools/stitching-directives	Patch
@graphql-tools/url-loader	Patch
@graphql-tools/graphql-tag-pluck	Patch
@graphql-tools/load	Patch
@graphql-tools/merge	Patch
@graphql-tools/relay-operation-optimizer	Patch
@graphql-tools/resolvers-composition	Patch
@graphql-tools/schema	Patch
@graphql-tools/apollo-engine-loader	Patch
@graphql-tools/code-file-loader	Patch
@graphql-tools/git-loader	Patch
@graphql-tools/github-loader	Patch
@graphql-tools/graphql-file-loader	Patch
@graphql-tools/json-file-loader	Patch
@graphql-tools/module-loader	Patch
@graphql-tools/prisma-loader	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[theguild-bot]

The latest changes of this PR are available as alpha in npm (based on the declared changesets):

@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]
@graphql-tools/[email protected]

[yaacovCR]

I'm also not really sure about which packages to bump to v8, I erred on bumping more rather than less.

[yaacovCR]

Right now, implementing as non-backwards compatible to v14/v15. Let's get it working first and see where we end up.

[gmac]

Hey, in v8, can we set batch to true on merged type config? Feels like the better default at this point.

[yaacovCR]

That would be smart! this PR for now is just for defer stream, possibly we need to create another release branch for v8 to work on other PRs...

[gmac]

Wooo, in-house pubsub? That’s exciting.

[yaacovCR]

Status update:

Good news -- after applying defer/stream fix for concurrent next calls via patch-package (https://github.com/graphql/graphql-js/pull/2975), significant drop in failing tests!

A good portion of the remaining failing tests are due to pending work on batch-delegate support.

[yaacovCR]

The work so far has had a lot of false starts that made the commit history difficult to follow. The latest force push updates the commit hx to be mostly by package, more granular when possible.

In theory, I will try to:

1. break down the larger commits into smaller pieces to make it even easier to follow.
2. separate out the changes that can be merged into v7 separately from defer/stream, like the new query planner.

The higher priorities are: (A) documenting each breaking changes with upgrade instructions, and documenting the most salient non-breaking changes (B) actually getting defer/stream working....

(A) and (B) are of much higher priority than 1. and 2.

[yaacovCR]

Status update:

1. We are now successfully using @defer to implement batch execution throughout the code base. The the slowest operation in a batch no longer holds up data from the rest.
2. mapAsyncIterator has been rewritten to rely on repeaters, as we were having some problems with resolution order with the implementation from graphql-js (see https://github.com/repeaterjs/repeater/issues/48#issuecomment-748275030). The implementation from graphql-js has undergone some recent changes that may or may not help -- in theory, we could contemplate switching back with the release of those changes, but repeaters seem like a safer bet for now.
3. All remaining failing tests are expected errors related to as-yet-implemented batch-delegate conversion.

[yaacovCR]

Status update: batch-delegate still not yet converted, but prep work in progress, all tests now passing

[sshevlyagin]

So... is this done? https://github.com/ardatan/graphql-tools/pull/4796 :D

[yaacovCR]

Not yet :( still can't stitch

[maxbol]

Hi @yaacovCR ! Do you know if this will solve other async/persistent connections, like @n1ru4l 's live query pattern, in stitched schemas?

I would love to assist with this goal in any way possible!