feat: add proxy support via global-agent

[miraclx]

Uses https://github.com/gajus/global-agent to aid proxying

Closes: https://github.com/miraclx/freyr-js/issues/95 Closes: https://github.com/miraclx/freyr-js/issues/120

[miraclx]

@Goooler, can you test this?

[Goooler]

How can I run this on local?

[miraclx]

$ git clone https://github.com/miraclx/freyr-js
$ cd freyr-js
$ git checkout proxy-support
$ npm install

# ensure the HTTP_PROXY, HTTPS_PROXY envs are set

$ node cli.js <URL>

Note that instead of freyr <URL>, since you're in development, you use node cli.js

[Goooler]

snebula@MBP  ~/IdeaProjects/freyr-js   proxy-support ±  node cli.js https://music.apple.com/cn/album/the-four-seasons-concerto-no-1-in-e-la-primavera/697328812\?i\=697329513
    ____
   / __/_______  __  _______
  / /_/ ___/ _ \/ / / / ___/
 / __/ /  /  __/ /_/ / /
/_/ /_/   \___/\__, /_/
              /____/ v0.5.0

freyr v0.5.0 - (c) Miraculous Owonubi <[email protected]>
-------------------------------------------------------------
Checking directory permissions...[done]
[https://music.apple.com/cn/album/the-four-seasons-concerto-no-1-in-e-la-primavera/697328812?i=697329513]
 [•] Identifying service...[Apple Music]
 [•] Checking authentication...[authenticated]
 Detected [track]
 Obtaining track metadata...[done]
  ➤ Title: The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro
  ➤ Album: Vivaldi: The Four Seasons
  ➤ Artist: 索菲娅·穆特, 赫伯特・冯・卡拉扬 & 维也纳爱乐
  ➤ Year: 1984
  ➤ Playtime: 03:18
 [•] Collating...
 • [01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro]
    | ➤ Collating sources...
    |  ➤ [•] YouTube Music...(failed: [read ECONNRESET])
    |  ➤ [•] YouTube...[success, found 13 sources]
    | ➤ Awaiting audiofeeds...[done]
    | [✓] Got album art
(node:39626) [DEP0123] DeprecationWarning: Setting the TLS ServerName to an IP address is not permitted by RFC 6066. This will be ignored in a future version.
(Use `node --trace-deprecation ...` to show where the warning was created)
    | {⯈} @meta{1/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{2/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{3/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{4/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{5/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{6/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{7/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{8/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{9/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | {⯈} @meta{10/10}: [ECONNRESET] (Client network socket disconnected before secure TLS connection was established)
    | [✕] Failed to get raw media stream [ECONNRESET]
 [•] Download Complete
 [•] Embedding Metadata...
  • [✕] 01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro [apple_music:track:697329513] (failed: Error downloading raw audio [An error occurred while collecting and parsing information from the URL: Client network socket disconnected before secure TLS connection was established])
[•] Collation Complete
============ Stats ============
 [•] Runtime: [19.7s]
 [•] Total queries: [01]
 [•] Total tracks: [01]
     » Skipped: [00]
     ✓ Passed:  [00]
     ✕ Failed:  [01]
 [•] Output directory: [.]
 [•] Cover Art: cover.png (640x640)
 [•] Total Output size: 0.00 B
 [•] Total Network Usage: 0.00 B
     ♫ Media: 0.00 B
     ➤ Album Art: 0.00 B
 [•] Output bitrate: 320k
===============================

[miraclx]

Halfway through, that's good.. Now to figure out this last part.

[miraclx]

Hm.. I'm not sure that this is a freyr issue. Can you make sure https_proxy isn't a "http" address? (I mean, https_proxy if specified, should be a "https" url, not "http")

[Goooler]

 [•] Collating...
 • [01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro]
    | ➤ Collating sources...
    |  ➤ [•] YouTube Music...(failed: [Unsupported `GLOBAL_AGENT.HTTP_PROXY` configuration value: URL protocol must be "http:".])
    |  ➤ [•] YouTube...(failed: [Unsupported `GLOBAL_AGENT.HTTP_PROXY` configuration value: URL protocol must be "http:".])
 [•] Download Complete
 [•] Embedding Metadata...
  • [✕] 01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro [apple_music:track:697329513] (failed: Failed collecting sources [Unsupported `GLOBAL_AGENT.HTTP_PROXY` configuration value: URL protocol must be "http:".])

[miraclx]

What is the output of;

$ echo $http_proxy

[Goooler]

http://127.0.0.1:7890

[miraclx]

global-agent seems to think otherwise.

https://github.com/gajus/global-agent/blob/3735fa1cac1a6680caa5afd853edba38674d8f27/src/utilities/parseProxyUrl.ts#L16-L18

[Goooler]

Yep, may use socket5 protocol?

[miraclx]

I'll try to reproduce this issue on my end and let you know the next steps.

[miraclx]

Just noticed that global-agent doesn't support the lowercase http_proxy. Can you try HTTP_PROXY.

export HTTP_PROXY=http://127.0.0.1:7890
node cli <SONG_URL>

[Goooler]

export https_proxy=http://127.0.0.1:7890 http_proxy=http://127.0.0.1:7890
export HTTP_PROXY=http://127.0.0.1:7890 HTTPS_PROXY=http://127.0.0.1:7890

 [•] Collating...
 • [01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro]
    | ➤ Collating sources...
    |  ➤ [•] YouTube Music...(failed: [read ECONNRESET])
    |  ➤ [•] YouTube...[success, found 21 sources]
    | ➤ Awaiting audiofeeds...(failed: [Command failed with ENOENT: /Users/snebula/IdeaProjects/freyr-js/node_modules/youtube-dl-exec/bin/youtube-dl --socket-timeout 20 --no-cache-dir --dump-single-json -- cW3RwYZWoxs
spawn /Users/snebula/IdeaProjects/freyr-js/node_modules/youtube-dl-exec/bin/youtube-dl ENOENT])
 [•] Download Complete
 [•] Embedding Metadata...
  • [✕] 01 The Four Seasons, Concerto No. 1 in E (La primavera/ Spring) RV269 (Op. 8 No. 1): I. Allegro [apple_music:track:697329513] (failed: Error while collecting sources feeds)
[•] Collation Complete

[miraclx]

I've spent so much time on this, but can't seem to reproduce the issue. 😪 Unfortunately ExpressVPN has no servers in China for me to test proxying with.

In my local patch tests, axios detects the proxy options, but when downloading, discards it for some reason. And got doesn't get the memo at all. I'll investigate on this line, but so far, idk..

If we don't have this figured out soon, I'll have to cut v0.6.0 as-is and leave this for later.

[Goooler]

Ok, still big thanks to your work ❤️

[SpinningK]

I'm from China, too. I repeated steps in this issue and got this output:

freyr v0.5.0 - (c) Miraculous Owonubi <[email protected]>
-------------------------------------------------------------
Checking directory permissions...[done] 
[https://open.spotify.com/track/7EoyERNueJuFlQbyrRB0US]
 [•] Identifying service...[Spotify]
 [•] Checking authentication...[authenticated]
 Detected [track]
 Obtaining track metadata...(failed: [Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:217:20)]) 
 [i] An error occurred while processing the query
============ Stats ============
 [•] Runtime: [2.1s]
 [•] Total queries: [01]
 [•] Total tracks: [00]
     » Skipped: [00]
     ✓ Passed:  [00]
     ✕ Failed:  [00]
 [•] Output directory: [.]
 [•] Cover Art: cover.png (640x640)
 [•] Total Output size: 0.00 B
 [•] Total Network Usage: 0.00 B
     ♫ Media: 0.00 B
     ➤ Album Art: 0.00 B
 [•] Output bitrate: 320k
===============================
superagent: double callback bug

[github-actions[bot]]

---

🐋 🤖

A docker image for this PR has been built!

docker pull freyrcli/freyrjs-git:pr-143

Latest Master
This Patch

---

What's this?

This docker image is a self-contained sandbox that includes all the patches made in this PR. Allowing others to easily use your patches without waiting for it to get merged and released officially.

For more context, see https://github.com/miraclx/freyr-js#docker-development.