@apollo/gateway: Abstract the transport layer

[orefalo]

Put simply,

Unless I missed something, you should abstract the transport rather that relying on node-fetch.

Sincerely,

[abernix]

You haven't missed anything and this is certainly something we want to do.

I'm curious what your use-cases and thoughts are about this? In other words, what are you trying to do? We're interested in hearing details to make sure we land on the right abstractions! 😄

[orefalo]

Hi @abernix,

HTTP is a relatively weak protocol when to comes to resilience and discovery. It requires lots of stitching and extra components. As the platform scales, the number of components grows along with the complexities and failure points.

Using different communication technologies can greatly simplify this work. For instance, systems such as NATS take care of the addressing, load-balancing & fail over seamlessly.

I like to keep things simple and stupid. ;-)

Sincerely,

[lennyburdette]

I'd like to add two specific requests for the transport layer.

1. Respect http(s)_proxy and no_proxy environment variables. We implement a proxy for request from our DCs to the wider internet. We can't reach Google Cloud Storage for managed federation config without going through the proxy.

   (A temporary solution is supporting a custom Agent in cachedFetcher, similar to the requestAgent option for engine config.)

2. Support unix: socket URLs. We use Envoy sidecars to manage service-to-service traffic, and node-fetch will not support unix: sockets by design. I can side-step this issue with a custom GraphQLDataSource, but that's a lot of extra code I'd rather not write. :grin:

@abernix @zionts

[ronan]

Our use case for this is to add our custom CA to https requests. We can use buildService to do this for queries but the introspection step fails because that doesn't use the willSendRequest modifier.

[GCHQDeveloper500]

I also need to be able to customise the transport layer (in my case, similar to others here, I need to set a custom agent). As per @ronan's comments above, is there a specific reason that we couldn't just have the buildService definitions be responsible for sending the introspection requests too?

I understand not wanting to couple this too tightly to the node-fetch api (reproduced here for readability, highlighting that some of it is bespoke to that API rather than being part of the official fetch definition) - from the node-fetch documentation;

{
    // These properties are part of the Fetch Standard
    method: 'GET',
    headers: {},        // request headers. format is the identical to that accepted by the Headers constructor (see below)
    body: null,         // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream
    redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect
    signal: null,       // pass an instance of AbortSignal to optionally abort requests
 
    // The following properties are node-fetch extensions
    follow: 20,         // maximum redirect count. 0 to not follow redirect
    timeout: 0,         // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead.
    compress: true,     // support gzip/deflate content encoding. false to disable
    size: 0,            // maximum response body size in bytes. 0 to disable
    agent: null         // http(s).Agent instance or function that returns an instance (see below)
}

however - I think the ability to customise some of these options is going to come up quite a lot.

If passing those options in via fetchOptions (as per Apollo link) isn't considered a viable solution, and there is a good reason not to use the buildService config already defined per service or that is a long way off, how about something like this for now;

const gateway = new ApolloGateway({
  ...
  introspectionFetcher({ serviceName, url, fetchOpts }) {
    // this must return a `Promise<Response>`
    return fetch(url, {
      ...fetchOpts, 
      agent: https.Agent({...})
    })
  }
});

I'd be happy to help out with the PR if any of the above is considered acceptable.

[ovesco]

Hey personally I'm working on an application based on moleculer and as such I'd love to be able to use Nats as transport layer.

This could allow better integration of Apollo with any kind of microservice framework. That would be awesome

[ujwal-setlur]

I am also looking to use moleculer with nats. Exact same use case as @ovesco

Any ideas on how this can be achieved?

[alfaproject]

Our use case is to use AWS API to call lambdas directly instead of having an API Gateway in front of them