axios and https proxy - is it possible to revert request lib drop?

[toaster74]

Describe the bug Axios does not work with corporate https proxies that are tunnelling https requests through http-CONNECT. All workflows worked with alternative request lib, that has been dropped in 1.0 (N8N_USE_DEPRECATED_REQUEST_LIB).

Is it possible to give again access to the request lib?

To Reproduce Steps to reproduce the behavior:

1. Create a http node and retrieve https://www.google.com and use corporate proxy (squid) via http_proxy and https_proxyenvironment variables
2. Click on Execute
3. It gets an error "Bad Gateway"

Expected behavior The html content of www.google.com should be retrieved.

Environment (please complete the following information):

• OS: docker
• n8n Version 1.0.4
• Database system : postgresql
• Operation mode : main

Additional context This behaviour has been documented by many sites (https://github.com/axios/axios/issues/4766).

[Joffcom]

Hey @toaster74,

Not at the moment, This is something we will likely look into soon but from our own testing using https_proxy works if you have it configured to use SSL. If you are using plain http and it has to use CONNECT it will fail, There are also a few threads on our forum about this as well.

For now if you rely on a proxy it would be worth staying on a pre v1 release and maybe pop a note on one of the other existing forum requests about this so we can try and keep the requests for this in the same place.

[Joffcom]

I have created N8N-6908 as our internal ticket to look at fixing the issue in Axios.

[biankasyo]

I have found a temporary solution to the issue. I replaced the http request node with a code node and ran the following JavaScript code. Please note that before executing this code, you will need to allow the corresponding extension library in the Docker environment variables and install the required library using the command node or command line.

const fetch = require('node-fetch');
const HttpsProxyAgent = require('https-proxy-agent');

const targetURL = 'URL_TO_YOUR_HTTPS_TARGET';
const proxyURL = 'URL_TO_YOUR_HTTP_PROXY';
const agent = new HttpsProxyAgent(proxyURL);

return new Promise((resolve, reject) => {
    fetch(targetURL, { agent })
      .then(res => res.text())
      .then(body => {
          items[0].json = {"data": body};
          resolve(items);
      })
      .catch(err => {
          console.error('Error:', err);  // More detailed error information
          reject(err);
      });
});

[Zetanova]

The current workaround is to use a httpS-proxy, where axios bug does not exist.

@Joffcom How to add the private CA to the n8n docker image without created a own image, mount it under "/etc/ssl/certs/nats-ca.pem" ?

[Zetanova]

@Joffcom I tried now squid proxy over https_port

In squids this is a TLS enabled port that assumes a direct TLS connection from the client. The hostname is taken over SNI value in the tls-helo packet and not over the proxy connect method. By default squid does not decrypt traffic and uses its splice mode to establish the connection.

I tried to add the used proxy CA into NODE_EXTRA_CA_CERTS and made my own image and extended /etc/ssl/certs/ca-certificates.crt

mozilla and curl from inside the container do not have any issues with the httpS proxy

The http request node (axios) seams not to use TLS SNI. It tries simply to connect to the https proxy port and errors because a hostname mismatch. Squid does not get the hostname supplied over SNI and replies with an error page and its set ssl cert.

[Zetanova]

It seams that axios/n8n/nodejs/openssl in the dcoker images uses the SSLv3 (legacy should not be used) Stack trace:

NodeApiError: write EPROTO 68ABFA8117730000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354: at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/HttpRequest/V3/HttpRequestV3.node.js:1571:35) 
...

But I could not test/configure the minVersion to tls1.2

Related issue: https://github.com/axios/axios/issues/2767

[kangschampagne]

hi~@Joffcom Has there been any recent progress on this issue? Despite configuring the appropriate http_proxy and https_proxy within Docker, the requests originating from the n8n container are not utilizing the CONNECT method. As a result, all functionalities dependent on HTTPS are failing to operate correctly due to the inability to properly use the proxy.