Subgraph support for query batching

[garypen]

This project is an extension of the existing work to support client side batching in the router. The current implementation is experimental and is publicly documented. The additional work to enable batching requests to subgraphs is captured in this issue. Currently the concept of a batch is preserved until the end of the RouterRequest processing. At this point we convert each batch request item into a separate SupergraphRequest. These are then planned and executed concurrently within the router and re-assembled into a batch when they complete. It's important to note that, with this implementation, the concept of a batch, from the perspective of an executing router, now disappears and each request is planned and executed separately. This extension will modify the router so that the concept of a batch is preserved, at least outwardly, so that multiple subgraph requests are "batched" (in exactly the same format as a client batch request) for onward transmission to subgraphs. The goal of this work is to provide an optimisation by reducing the number of round-trips to a subgraph from the router. Additionally, the work will address an unresolved issue from the existing experimental implementation and promote the existing implementation from experimental to fully supported.

Fixes #2002

---

Checklist

Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.

• [x] Changes are compatible[^1]
• [ ] Documentation[^2] completed
• [ ] Performance impact assessed and acceptable
• Tests added and passing[^3]
  • [ ] Unit Tests
  • [ ] Integration Tests
  • [ ] Manual Tests

Exceptions

Note any exceptions here

Notes

[^1]: It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this. [^2]: Configuration is an important part of many changes. Where applicable please try to document configuration examples. [^3]: Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions.

[github-actions[bot]]

@garypen, please consider creating a changeset entry in /.changesets/. These instructions describe the process and tooling.

[router-perf[bot]]

CI performance tests

• [ ] reload - Reload test over a long period of time at a constant rate of users
• [ ] events_big_cap_high_rate_callback - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity using callback mode
• [ ] events_without_dedup_callback - Stress test for events with a lot of users and deduplication DISABLED using callback mode
• [ ] large-request - Stress test with a 1 MB request payload
• [x] const - Basic stress test that runs with a constant number of users
• [ ] no-graphos - Basic stress test, no GraphOS.
• [ ] step-jemalloc-tuning - Clone of the basic stress test for jemalloc tuning
• [ ] events - Stress test for events with a lot of users and deduplication ENABLED
• [ ] events_callback - Stress test for events with a lot of users and deduplication ENABLED in callback mode
• [ ] events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
• [ ] events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
• [ ] xxlarge-request - Stress test with 100 MB request payload
• [ ] xlarge-request - Stress test with 10 MB request payload
• [x] step - Basic stress test that steps up the number of users over time

[smyrick]

Merge request to log the request body in debug mode: https://github.com/apollographql/router/pull/4840