[Bug]: docker use HTTP_PROXY env request error

[ilwzmf]

Describe the issue

docker里配置了HTTP_PROXY和HTTPS_PROXY访问任何地址都报错 我需要使用代理才能访问google和部分podcasts 此代理在其他docker使用正常，同样的地址使用此代理在chrome访问正常

Sorry for the bad English, the following is google translation

HTTP_PROXY and HTTPS_PROXY are configured in docker to access any address and report an error I need to use a proxy to access google and some podcasts This proxy is used normally in other dockers, and the same address uses this proxy to access normally in chrome

Steps to reproduce the issue

请求已经到了代理服务器，但返回的都是错误，我不确定是否传的参数有问题

以下为log: Failed Error: Request failed with status code 400 at createError (/node_modules/axios/lib/core/createError.js:16:15) at settle (/node_modules/axios/lib/core/settle.js:17:12) at IncomingMessage.handleStreamEnd (/node_modules/axios/lib/adapters/http.js:322:11) at IncomingMessage.emit (node:events:525:35) at endReadableNT (node:internal/streams/readable:1358:12) at processTicksAndRejections (node:internal/process/task_queues:83:21) { config: { transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, adapter: [Function: httpAdapter], transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, validateStatus: [Function: validateStatus], headers: { Accept: 'application/json, text/plain, /', 'User-Agent': 'axios/0.26.1', host: 'openlibrary.org' }, method: 'get', url: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', data: undefined }, request: <ref *1> ClientRequest { _events: [Object: null prototype] { abort: [Function (anonymous)], aborted: [Function (anonymous)], connect: [Function (anonymous)], error: [Function (anonymous)], socket: [Function (anonymous)], timeout: [Function (anonymous)], finish: [Function: requestOnFinish] }, _eventsCount: 7, _maxListeners: undefined, outputData: [], outputSize: 0, writable: true, destroyed: false, _last: true, chunkedEncoding: false, shouldKeepAlive: false, maxRequestsOnConnectionReached: false, _defaultKeepAlive: true, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, strictContentLength: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, _closed: false, socket: Socket { connecting: false, _hadError: false, _parent: null, _host: 'v2fly-uncn', _closeAfterHandlingError: false, _readableState: [ReadableState], _events: [Object: null prototype], _eventsCount: 7, _maxListeners: undefined, _writableState: [WritableState], allowHalfOpen: false, _sockname: null, _pendingData: null, _pendingEncoding: '', server: null, _server: null, parser: null, _httpMessage: [Circular *1], [Symbol(async_id_symbol)]: 55362, [Symbol(kHandle)]: [TCP], [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: null, [Symbol(kBuffer)]: null, [Symbol(kBufferCb)]: null, [Symbol(kBufferGen)]: null, [Symbol(kCapture)]: false, [Symbol(kSetNoDelay)]: false, [Symbol(kSetKeepAlive)]: true, [Symbol(kSetKeepAliveInitialDelay)]: 60, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0, [Symbol(RequestTimeout)]: undefined }, _header: 'GET https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93 HTTP/1.1\r\n' + 'Accept: application/json, text/plain, /\r\n' + 'User-Agent: axios/0.26.1\r\n' + 'host: openlibrary.org\r\n' + 'Connection: close\r\n' + '\r\n', _keepAliveTimeout: 0, _onPendingData: [Function: nop], agent: Agent { _events: [Object: null prototype], _eventsCount: 2, _maxListeners: undefined, defaultPort: 80, protocol: 'http:', options: [Object: null prototype], requests: [Object: null prototype] {}, sockets: [Object: null prototype], freeSockets: [Object: null prototype] {}, keepAliveMsecs: 1000, keepAlive: false, maxSockets: Infinity, maxFreeSockets: 256, scheduling: 'lifo', maxTotalSockets: Infinity, totalSocketCount: 1, [Symbol(kCapture)]: false }, socketPath: undefined, method: 'GET', maxHeaderSize: undefined, insecureHTTPParser: undefined, path: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', _ended: true, res: IncomingMessage { _readableState: [ReadableState], _events: [Object: null prototype], _eventsCount: 4, _maxListeners: undefined, socket: [Socket], httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, rawHeaders: [Array], rawTrailers: [], aborted: false, upgrade: false, url: '', method: null, statusCode: 400, statusMessage: 'Bad Request', client: [Socket], _consuming: false, _dumped: false, req: [Circular *1], responseUrl: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', redirects: [], [Symbol(kCapture)]: false, [Symbol(kHeaders)]: [Object], [Symbol(kHeadersCount)]: 14, [Symbol(kTrailers)]: null, [Symbol(kTrailersCount)]: 0, [Symbol(RequestTimeout)]: undefined }, aborted: false, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, reusedSocket: false, host: 'v2fly-uncn', protocol: 'http:', _redirectable: Writable { _writableState: [WritableState], _events: [Object: null prototype], _eventsCount: 3, _maxListeners: undefined, _options: [Object], _ended: true, _ending: true, _redirectCount: 0, _redirects: [], _requestBodyLength: 0, _requestBodyBuffers: [], _onNativeResponse: [Function (anonymous)], _currentRequest: [Circular *1], _currentUrl: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', [Symbol(kCapture)]: false }, [Symbol(kCapture)]: false, [Symbol(kBytesWritten)]: 0, [Symbol(kEndCalled)]: true, [Symbol(kNeedDrain)]: false, [Symbol(corked)]: 0, [Symbol(kOutHeaders)]: [Object: null prototype] { accept: [Array], 'user-agent': [Array], host: [Array] }, [Symbol(kUniqueHeaders)]: null }, response: { status: 400, statusText: 'Bad Request', headers: { 'content-length': '264', connection: 'keep-alive', 'content-type': 'text/html', date: 'Sat, 24 Dec 2022 10:11:24 GMT', 'keep-alive': 'timeout=4', 'proxy-connection': 'keep-alive', server: 'nginx/1.18.0 (Ubuntu)' }, config: { transitional: [Object], adapter: [Function: httpAdapter], transformRequest: [Array], transformResponse: [Array], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, validateStatus: [Function: validateStatus], headers: [Object], method: 'get', url: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', data: undefined }, request: <ref *1> ClientRequest { _events: [Object: null prototype], _eventsCount: 7, _maxListeners: undefined, outputData: [], outputSize: 0, writable: true, destroyed: false, _last: true, chunkedEncoding: false, shouldKeepAlive: false, maxRequestsOnConnectionReached: false, _defaultKeepAlive: true, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, strictContentLength: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, _closed: false, socket: [Socket], _header: 'GET https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93 HTTP/1.1\r\n' + 'Accept: application/json, text/plain, /\r\n' + 'User-Agent: axios/0.26.1\r\n' + 'host: openlibrary.org\r\n' + 'Connection: close\r\n' + '\r\n', _keepAliveTimeout: 0, _onPendingData: [Function: nop], agent: [Agent], socketPath: undefined, method: 'GET', maxHeaderSize: undefined, insecureHTTPParser: undefined, path: 'https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93', _ended: true, res: [IncomingMessage], aborted: false, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, reusedSocket: false, host: 'v2fly-uncn', protocol: 'http:', _redirectable: [Writable], [Symbol(kCapture)]: false, [Symbol(kBytesWritten)]: 0, [Symbol(kEndCalled)]: true, [Symbol(kNeedDrain)]: false, [Symbol(corked)]: 0, [Symbol(kOutHeaders)]: [Object: null prototype], [Symbol(kUniqueHeaders)]: null }, data: '\r\n' + '

\r\n' + '\r\n' + '

400 Bad Request

\r\n' + 'The plain HTTP request was sent to HTTPS port\r\n' + '

---

nginx/1.18.0 (Ubuntu)\r\n' + '\r\n' + '\r\n' }, isAxiosError: true, toJSON: [Function: toJSON] } [2022-12-24 18:11:24] ERROR: OpenLib Search Error 404

我在bash里可以正确访问，我将地址从https://openlibrary.org//search.json?title=%E4%B8%89%E4%BD%93改为了https://openlibrary.org/search.json?title=%E4%B8%89%E4%BD%93 见如下截图

Audiobookshelf version

v2.2.11

How are you running audiobookshelf?

Docker

[advplyr]

I'm not sure what to do here so will need more information on specifically what needs to be done.

[ilwzmf]

It's maybe a bug about axios.

Using the following code works fine. It might be better to use unity interceptor.

const Axios = require("axios"); const HttpsProxyAgent = require("https-proxy-agent");

const httpsAgent = new HttpsProxyAgent(http://${YOUR_HOST}:${YOUR_PORT}); const axios = Axios.create({ proxy:false, httpsAgent })

[advplyr]

What would "YOUR_HOST" and "YOUR_PORT" be in your example? How would Abs get that information? Why did you choose to use "http://" for the HttpsProxyAgent?

[ilwzmf]

The environment variable HTTP_PROXY is common in most dockers. Axios has automatically used the environment variable HTTP_PROXY, but axios has a bug when requesting https links through http proxy.

eg:HTTP_PROXY=http://192.168.1.2:9001 const httpsAgent = new HttpsProxyAgent(process.env.HTTP_PROXY);

I use v2fly, supports socks5 or http proxy. After I tested, I found that axios may not support socks5.

[geekidentity]

Has this issue been fixed?

[advplyr]

I don't understand the issue so I added the help wanted tag

[geekidentity]

@advplyr What he means is whether it can support the HTTP_PROXY and HTTPS_PROXY environment variables of the Linux system. This is required in the scenario of using VPN.

[ykelvis]

Support for proxy would be very useful, as there are usecases where the host is behind firewall, requires some proxy setting to access internet.

[zhzy0077]

Looked into it in detail. 2 items on the way:

• ssrf-req-filter issue HTTP Proxy is not well supported in HTTPS connection. · Issue #42 · y-mehta/ssrf-req-filter
• upgrade to axios 1.x version. version bisect says the fix is between 0.27.2 and 1.0.0. Changes are too much in middle.

PR at #2666

[asseywang]

I'm having the same problem, has this been resolved now?

[2024-03-04 13:49:55.073] ERROR: [podcastUtils] getPodcastFeed Error Error: Call to 10.1.1.1 is blocked. at agent.createConnection (/node_modules/ssrf-req-filter/lib/index.js:31:13) at Agent.createSocket (node:_http_agent:340:26) at Agent.addRequest (node:_http_agent:288:10) at new ClientRequest (node:_http_client:337:16) at Object.request (node:http:103:10) at RedirectableRequest._performRequest (/node_modules/follow-redirects/index.js:326:24) at new RedirectableRequest (/node_modules/follow-redirects/index.js:100:8) at Object.request (/node_modules/follow-redirects/index.js:532:14) at dispatchHttpRequest (/node_modules/axios/lib/adapters/http.js:262:25) at new Promise (<anonymous>) (podcastUtils.js:265)