Support proxies

[totof129]

download should call got module using tunnel agent when user is behind a corporate proxy

[shinnn]

PR for got module is very welcome. https://github.com/sindresorhus/got/issues/7#issuecomment-68736680

[alfonso-presa]

:+1:

[shinnn]

@alfonso-presa Have you read my comment?

[alfonso-presa]

Yes. I did what I could in the PR I just submitted :-).

Truth is that this was working perfectly last year but when 4.0.0 got released it stated breaking imagemin on our build pipeline... I'm trying my best to fix the problem without having the full node support proxies natively.

[kevva]

@alfonso-presa, yea, I used the request module here before but decided to switch to got instead because of performance. I don't think the proxy stuff belongs in got anyway.

[alexpaluzzi]

Any progress here? Since this busted, we can no longer run npm install on most of our production apps.

[kevva]

Could someone test this?

[alfonso-presa]

I'm currently traveling, but I'll test it ASAP

[alexpaluzzi]

@kevva Doesn't seem to be working. I just tested (making sure any cached copies were nuked out). It is only using 4.2.0.

Still failing to download through https.

It's worth mentioning, I'm not technically using a "proxy." I do, however, have SSL inspection down the line. Everything worked perfectly on 4.1.2.

[alexpaluzzi]

@kevva I'm here to test if you need me, I'll be here all day. Whatever I can do to help.

[teehemkay]

FWIW just tried the current master with the following example:

var Download = require('./');

new Download({mode: '755'})
    .get('https://github.com/kevva/elm-bin/blob/2.0.0/vendor/win/elm-platform-win.tar.gz')
    .dest('downloads')
    .run(function(err, files) {
        if (err) console.log(err);
        console.log(files);
    });

Produced the following:

{ [GotError: Request to https://github.com/kevva/elm-bin/blob/2.0.0/vendor/win/elm-platform-win.tar.gz failed]
  nested: { [Error: tunneling socket could not be established, cause=Parse Error] code: 'ECONNRESET' },
  code: 'ECONNRESET'}

[mbrand]

Any news about this? We still have problems with imagemin building because of this. I tried switching from 'got' to 'request' which worked fine in our environment. Was it that much of a performance problem?

[marc-guenther]

So, when I try to download via https like this:

var Download = require('download');
new Download().get('https://raw.github.com/imagemin/optipng-bin/v3.0.3/vendor/osx/optipng').dest('tmp').run()

it works, but when I set a proxy:

https_proxy=http://mylocalproxy:3128 node download.js

and look at the network traffic, I see that it tells the proxy to connect to port 80:

CONNECT raw.github.com:80 HTTP/1.1

but it then continues to talk SSL, which of course confuses the server completely.

It works when I specify the port in the github url above: https://raw.github.com:443/...

This seems to be a problem in tunnel-agent, which fails to set the defaultPort to 443 for it's https agents... (maybe this issue?)

[marc-guenther]

mikeal/tunnel-agent#15 fixed this problem for us.

[shinnn]

@marc-guenther Great work!

Even if https://github.com/mikeal/tunnel-agent/pull/15 is merged, we should add some tests to check if this module can handle proxy setting correctly. Currently this project doesn't have any tests related to proxy. https://github.com/kevva/download/blob/429f255af0a30609e19b547ad603cacbba8653da/test/test.js

Could anyone help us? I'm not very good at network and HTTP/HTTPS.

[wiibaa]

@marc-guenther sorry for the silly question, but does your fix on tunnel-agent works with the current release of download or do you need to include tunnel-agent explicitly as it was proposed in the current open pull request #65 ? I'm also stuck behind a corporate proxy trying to resolve dependencies such as https://github.com/imagemin/jpegtran-bin that use download internally to download the binaries and would like to try it out with your fix but I'm lost on how to do it in a simple way. Can you give me some clues ?

[marc-guenther]

@wiibaa well, we built the fixed tunnel-agent locally, and uploaded it to our Nexus as 0.4.1-hotfix version and added that our shrinkwrap. Since than we had no problems with proxy anymore... The #65 pull request basically tries to do the same thing (explicitly setting the port), but only for this specific module, as the real problem is in tunnel-agent

[marc-guenther]

@shinn Good idea. Unfortunately I don't know nothing about Node, so I will leave this for someone more knowledgeable :) Also, there are no tests whatsoever in tunnel-agent, so I don't really know where to put them....

[wiibaa]

@marc-guenther thanks for confirming that shrinkwrap is the only way. I will try this out soon

[marc-guenther]

New version of [email protected] has just been released. This contains the mikeal/tunnel-agent#15 fix. :-)

[shinnn]

@alexpaluzzi Can you check if https://github.com/kevva/download/issues/64#issuecomment-126704776 is fixed?

[plrthink]

I'm constantly getting this error when I set proxy property:

tunneling socket could not be established, cause=connect EHOSTUNREACH 0.0.31.144:80 - Local (10.10.0.5:56041)

And the code I use is

new Download({
      extract: true,
      proxy: 'http://101.127.219.182:8080',
    })

[lemures-t]

@plrthink I googled here since I had the same problem when trying to install gifsicle in terminal of OS X behind proxy. The weird IP 0.0.31.144 was also print. It is caused by the http_proxy set in ~/.bash_profile. The protocol (http e.g.) should be added before host:port. So the http_proxy should be protocal://host:port instead of host:port。It seems the command apt-get in ubuntu will also encounter this problem.Reference is here. Hope it helps.

[Jackbennett]

You'll need to handle self-signed certificates too.