Cargo build failed with spurious network error

[weiby3]

Problem error: failed to load source for a dependency on rand

Steps

$ cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
error: failed to load source for a dependency on `rand`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: 无法与服务器建立连接
; class=Os (2)

The chinese sentences above are "Unable to establish connection to server". I have check my firewall, cargo isn't blocked, and there is personal CA installed by Adguard, no proxy.

Notes OS: Windows 7 64bit, mingw64 with MSYS2 Output of cargo version: cargo 1.29.0

[alexcrichton]

Hm is this perhaps a network-level issue where github.com or github.com/rust-lang is blocked? Otherwise I'm not sure how we'd fix this as it seems Cargo can't connect :(

[weiby3]

I try git clone and curl https://github.com/rust-lang and they works fine. On my computer github.com can be accessed successfully, I have search the firewall and there aren't any block rule related to it. Maybe these are some useful information. I use HUORONG SECURITY as my anti-virus software, and if there is a program trying to connect to Internet, it will popup a windows and ask for permission. Strangely when I run cargo run --verbose there is no popup window, but if I run curl https://github.com/rust-lang there is a popup window, so it seems Cargo can't connect. Is it possible to get more error information?

[alexcrichton]

Hm so AFAIK there's a few different ways to touch the network on Windows, and it may be that Cargo, through libgit2, is using a different way that's may blocked or doesn't provide a popup? Can you temporarily disable your firewall/virus scanning to see if that fixes the issue?

[ddw360]

@alexcrichton Disabling antivirus, firewalls, etc - don't seem to work. I was able to do a cargo build in Windows 10 x64 but not in Windows Subsystem for Linux. Both have versions re: nightly, etc. The build fails when trying to retrieve from github dependencies. Command issued was "cargo build --release --target=wasm32-unknown-unknown"

[alexcrichton]

Hm interesting! So this probably isn't a firewall/antivirus issue if you've disabled all those and it still didn't work. It works on Windows though and not the WSL? What we use on Linux is far more bog-standard I think than the Windows implementation, so I'm surprised it wouldn't work there if it works on Windows... I wonder if that's a WSL bug?

[ddw360]

WSL environment info: Linux ZEPHYR 4.4.0-17134-Microsoft #471-Microsoft Fri Dec 07 20:04:00 PST 2018 x86_64 x86_64 x86_64 GNU/Linux PRETTY_NAME="Ubuntu 16.04.5 LTS" Architecture: x86_64

Don't know if it matters or not - but here are contents of Cargo.toml below and the git repos that the cargo build command is trying to pull for dependencies. when "cargo build --release --target=wasm32-unknown-unknown" is issued. Fails on trying to resolve the first "hdk" dependency.

[package] name = "chat" version = "0.1.0" authors = ["willem [email protected]"] edition = "2018" [dependencies] serde = "1.0" serde_json = "1.0" serde_derive = "1.0" machine-ip = "0.2.1" hdk = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } holochain_wasm_utils = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } holochain_core_types_derive = { git = "https://github.com/holochain/holochain-rust" , branch = "master" } [lib] path = "src/lib.rs" crate-type = ["cdylib"]

[alexcrichton]

Can you perhaps try strace? That may help illuminate which syscall is failing and may help tracking this down?

[ddw360]

@alexcrichton https://pastebin.com/uqcrHrzT

[weiby3]

Sorry for late response. I have tried to disable firewall and anti-virus software, but it still has this problem. Is it possible to expose the download directory filename and/or commands? Maybe try to do it by myself can solve this problem.

[ddw360]

@weiby3 @alexcrichton Weiby3 - I am curious if you are in actual Windows standard command line, powershell or a WSL environment when encountering issues. I haven't tested doing things in Powershell but do get the spurious network error in WSL.

[weiby3]

I have tried in cmd.exe and got the same problem. Windows 7 doesn't have WSL, so I can't try. Powershell also not work.

[alexcrichton]

@ddw360 do you perhaps have a proxy configured? It looks like an ntlm proxy or something like that, running on port 3128

[ddw360]

@alexcrichton don't think so - here is my evidence:

1. from Windows cmd prompt: netstat -abno | findstr /R "^.:3128.$" --> no results

2. Internet Explorer Settings - no proxy being used for internet connection

3. whatismyipaddress: Hostname: | cpe-76-170-98-202.socal.res.rr.com Services: None detected Proxy server not detected. Broadband Dynamic IP

4. netstat from bash (ubuntu 16.04) shows no use of proxy port 3128

5. /etc/resolv.conf shows no use of proxy - nameservers are all gateway addresses and then gateway pointed to publ ic dns/internet

6. git version is 2.7.4 (<< i am going to try to get a later version of git by compiling from source and see if that does the trick since cargo relies on git - since current ubuntu is 16.04).

I am going to try to move to WSL ubuntu 18.04 LTS and a higher git version and see if that changes things at all.

[weiby3]

If I try to run cargo via a proxy, it can "Updating registry https://github.com/rust-lang/crates.io-index", but it will get the following output:

$ https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
 Downloading winapi-x86_64-pc-windows-gnu v0.4.0
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)

Othrtwise, I try to install rust with msvc, it can download and build successfully.

c:\msys64\tmp\rust\guessing_game>cargo run
   Compiling winapi v0.3.6
   Compiling libc v0.2.46
   Compiling rand v0.4.3
   Compiling rand v0.3.22
   Compiling guessing_game v0.1.0 (C:\msys64\tmp\rust\guessing_game)
    Finished dev [unoptimized + debuginfo] target(s) in 7.72s
     Running `target\debug\guessing_game.exe`
Guess the number!
Please input your guess.
123
You guessed: 123

UPDATE: After download rust package via proxy, I run cargo without proxy, and it build successfully. Here are all the commands I have tried and their output:

$ cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
warning: spurious network error (2 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: 无法与服务器建立连接
; class=Os (2)
error: failed to load source for a dependency on `rand`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: 无法与服务器建立连接
; class=Os (2)

$ https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rand v0.3.22
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)

https_proxy=socks5://127.0.0.1:1080 cargo run --verbose
 Downloading rand v0.3.22
error: unable to get packages from source

Caused by:
  [35] SSL connect error (schannel: failed to receive handshake, SSL/TLS connection failed)

$ cargo run --verbose
   Compiling winapi-x86_64-pc-windows-gnu v0.4.0
   Compiling winapi v0.3.6
   Compiling libc v0.2.46
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=cd7a6ff3618fa678 -C extra-filename=-cd7a6ff3618fa678 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-x86_64-pc-windows-gnu-cd7a6ff3618fa678' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.6\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="minwindef"' --cfg 'feature="ntsecapi"' --cfg 'feature="profileapi"' --cfg 'feature="winnt"' -C metadata=db0e30f3a93487bc -C extra-filename=-db0e30f3a93487bc --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-db0e30f3a93487bc' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name build_script_build 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.46\build.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=f439455b17189f8e -C extra-filename=-f439455b17189f8e --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\build\libc-f439455b17189f8e' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-x86_64-pc-windows-gnu-cd7a6ff3618fa678\build-script-build`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\libc-f439455b17189f8e\build-script-build`
     Running `rustc --crate-name winapi_x86_64_pc_windows_gnu 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=b1bde479814dcaee -C extra-filename=-b1bde479814dcaee --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
     Running `C:\msys64\tmp\rust\guessing_game\target\debug\build\winapi-db0e30f3a93487bc\build-script-build`
     Running `rustc --crate-name libc 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.46\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="use_std"' -C metadata=df7ade2be09a8644 -C extra-filename=-df7ade2be09a8644 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --cap-lints allow`
     Running `rustc --crate-name winapi 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.6\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="minwindef"' --cfg 'feature="ntsecapi"' --cfg 'feature="profileapi"' --cfg 'feature="winnt"' -C metadata=7a3aa7d7c95390c7 -C extra-filename=-7a3aa7d7c95390c7 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'winapi_x86_64_pc_windows_gnu=C:\msys64\tmp\rust\guessing_game\target\debug\deps\libwinapi_x86_64_pc_windows_gnu-b1bde479814dcaee.rlib' --cap-lints allow --cfg 'feature="sspi"' --cfg 'feature="wincred"' --cfg 'feature="lsalookup"' --cfg 'feature="excpt"' --cfg 'feature="ktmtypes"' --cfg 'feature="basetsd"' --cfg 'feature="guiddef"' --cfg 'feature="vcruntime"' --cfg 'feature="subauth"' --cfg 'feature="windef"' --cfg 'feature="ntdef"' -l dylib=winapi_advapi32 -l dylib=winapi_credui -l dylib=winapi_kernel32 -l dylib=winapi_secur32 -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling rand v0.4.3
     Running `rustc --crate-name rand 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\rand-0.4.3\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="libc"' --cfg 'feature="std"' -C metadata=525ab3e86af69877 -C extra-filename=-525ab3e86af69877 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'winapi=C:\msys64\tmp\rust\guessing_game\target\debug\deps\libwinapi-7a3aa7d7c95390c7.rlib' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling rand v0.3.22
     Running `rustc --crate-name rand 'C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\rand-0.3.22\src\lib.rs' --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=028fc22332238648 -C extra-filename=-028fc22332238648 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'libc=C:\msys64\tmp\rust\guessing_game\target\debug\deps\liblibc-df7ade2be09a8644.rlib' --extern 'rand=C:\msys64\tmp\rust\guessing_game\target\debug\deps\librand-525ab3e86af69877.rlib' --cap-lints allow -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
   Compiling guessing_game v0.1.0 (file:///C:/msys64/tmp/rust/guessing_game)
     Running `rustc --crate-name guessing_game 'src\main.rs' --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=0ad3830cad379b28 -C extra-filename=-0ad3830cad379b28 --out-dir 'C:\msys64\tmp\rust\guessing_game\target\debug\deps' -C 'incremental=C:\msys64\tmp\rust\guessing_game\target\debug\incremental' -L 'dependency=C:\msys64\tmp\rust\guessing_game\target\debug\deps' --extern 'rand=C:\msys64\tmp\rust\guessing_game\target\debug\deps\librand-028fc22332238648.rlib' -L 'native=C:\Users\Administrator\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib'`
    Finished dev [unoptimized + debuginfo] target(s) in 16.01s
     Running `target\debug\guessing_game.exe`
Guess the number!
Please input your guess.
123
You guessed: 123

[ddw360]

@alexcrichton upgrading to WSL ubuntu 18.04, git 2.20.1, lastet rust and cargo nightly has not resolved. Could this be related to: https://github.com/rust-lang/cargo/issues/1180# in any way?

[alexcrichton]

This doesn't look related to certificate validation just yet, but your examples are also somewhat hard to follow for me :(. What caused the packages to actually be downloaded? It looks like in some cases connections to crates.io are disallowed but github is allowed. (I'm not really sure what's going on here...)

[weiby3]

The project comes from a tutorial. The following is Cargo.toml.

[package]
name = "guessing_game"
version = "0.1.0"
authors = ["account"]

[dependencies]
rand = "0.3.14"

[ddw360]

@alexcrichton I am trying to run the holochain stack and an example project See: https://developer.holochain.org/start.html https://github.com/holochain/holochat-rust

The "hc package" and "hc test" commands issue the following "cargo": "cargo build --release --target=wasm32=unknown-unknown"

and it fails - with SSL related issues - per the pastebin's I posted.

For me - what I have is problems specifically in Windows Subsystem for Linux. And pathing to cargo or hc are not the problem - they both come up correctly in bash.

In Windows 10 cmd.exe and powershell - no problems once I made sure that cargo and rustc were properly in the system path.

@weiby3 is likely having different root cause? But same resulting error - re: SSL and Spurious and git?

[d3sbar]

@alexcrichton I am trying different projects in Windows Ubuntu 18.04 WSL - i.e. actix/examples. So cloned down the repo, go to the "hello-world" example and do cargo run. Get spurious network error. Firewall is turned off, no antivirus running, etc.

$ cargo build
    Updating crates.io index
warning: spurious network error (2 tries remaining): [7] Couldn't connect to server; class=Net (12)
warning: spurious network error (1 tries remaining): [7] Couldn't connect to server; class=Net (12)
error: failed to load source for a dependency on `actix`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [7] Couldn't connect to server; class=Net (12)

[differui]

@d3sbar I stuck in same error. Fix that by removing [http_proxy] field in ~/.cargo/config.

[WindSoilder]

Got the same issue while Updating crate.io index... The error message is:

warning: spurious network error (2 tries remaining): HTTP parser error: stream ended at an unexpected time; class=Net (12)

But I think it's just caused by my Network problem...

I use wireshark to capture segments between my host and github(The ip address is 13.229.188.59).

From wireshark, I find the following phenomena:

1. github have tried to send many Retransmission for the same segment and,
2. my host is sending and sending back duplicate ACK.

~~It seems that my ACK can't reach github machine in time~~ It seems that it's just cause by network problem(like network congestion)

IMO, if there are stable Crates.io MIRROR exists in my region(Asia), the problem can be solved(by editing ~/.cargo/config).

Hope this information can help..

My OS: Mac 10.14.6 Cargo version: 1.36.0

[baburkin]

I got the same issue on Windows 7 and it looks like the reason behind that is cargo cannot find its way through our corporate NTLM proxy.

I was troubleshooting this with plain curl. When I try to run

curl https://github.com/rust-lang/crates.io-index

I get the response

curl: (56) Received HTTP code 407 from proxy after CONNECT

which implies that I need to authenticate in the proxy. By using curl with -vv option I can see that it tries to authenticate using NTLM:

...
Proxy-Authenticate: Negotiate
Proxy-Authenticate: NTLM

* Ignore NNNN bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT

Then what I did I added the following environment variables via Control Panel (the addresses are distorted on purpose):

HTTP_PROXY=http://ntlm-proxy.org.com:8080
HTTPS_PROXY=http://ntlm-proxy.org.com:8080
PROXY_USER=NT_DOMAIN\username
PROXY_PASS=the_password  <-- I know, not ideal, but see below

After that I was able to receive the crates index using the following command:

curl -vv --proxy-any -U : https://github.com/rust-lang/crates.io-index

It worked, it also worked with --proxy-ntlm instead of --proxy-any, but if I don't specify this parameter, I get the 407 error again. Instead of adding environment vars for user and password, I also was able to specify --proxy-user NT_DOMAIN\username and provide the password interactively - and it also worked.

So it looks like at the moment rust's cargo lacks support for NTLM proxy specifics, including authentication. Given that curl does support this, I think it is just a matter of adding this support in future versions.

Ideally, it would be nice to be able to to specify in ~/.cargo/config:

[http]
proxy = "host:port" # HTTP proxy to use for HTTP requests (defaults to none)
proxy-auth = "ntlm" # negotiate, basic, digest, etc... 
proxy-user = "username"
proxy-password = "password"
...          

[alexcrichton]

Thanks for the investigation @baburkin!

This is unfortunately a pretty long a sprawling issue, mind opening a focused issue purely on supporting proxy authentication (aka NTLM)?

[baburkin]

@alexcrichton - created Issue 7330 as a feature request. Would love to contribute.

[iav]

Have similar error trying to do a cross-compilation in docker. I got it in automatic builder on a Docker Hub, then on a wps in Amazon S3. This dockerfile (scenario) works on my pc. And it's not a "simple" network error, like dns not work, site unreacheable, proxy should be used, proxy auth etc. Problem occurs on machine free internet access, and target sites reacheable from same sessions with wget

error: (in docker container)

cargo --version
cargo 1.37.0 (9edd08916 2019-08-02)

uname -a
Linux b584dc1b7555 5.0.0-1012-aws #13-Ubuntu SMP Fri Aug 2 12:25:32 UTC 2019 armv7l GNU/Linux

# qemu-arm-static --version
qemu-arm version 4.1.0 (qemu-4.1.0-1.fc31)

outside container —  aws instance:
$ uname -a
Linux ip-172-30-0-82 5.0.0-1012-aws #13-Ubuntu SMP Fri Aug 2 12:25:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux


# cargo run --verbose
    Updating crates.io index
warning: spurious network error (2 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
warning: spurious network error (1 tries remaining): could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)
error: failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  could not read directory '/usr/local/cargo/registry/index/github.com-1ecc6299db9ec823/.git//refs': Value too large for defined data type; class=Os (2)

Start of Dockerfile to reproduce inside AWS t2.medium instance, image id ubuntu/images/hvm-ssd/ubuntu-disco-19.04-amd64-server-20190819 (ami-00290d66f0da25f73)

FROM multiarch/qemu-user-static as qemu
FROM arm32v7/rust:1.37-buster as rust
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
WORKDIR /
RUN git clone https://github.com/rustapp.git app
WORKDIR /app
RUN cargo build

[iav]

before run cross-platform image, qemu need to be added to system. Fast and easy for container runner docker run --rm --privileged multiarch/qemu-user-static:register --reset

[ehuss]

@iav can you file a new issue, that seems unrelated to this one.

[iav]

ok.

[kasvith]

The issue remains the same in my Windows 10 x64 setup too.

I have not set up proxies either.

toolchain stable-x86_64-pc-windows-msvc cargo version is cargo 1.38.0 (23ef9a4ef 2019-08-20)

$ cargo run
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to send request: The server name or address could not be resolved
; class=Os (2)
warning: spurious network error (1 tries remaining): failed to send request: The server name or address could not be resolved
; class=Os (2)
error: failed to load source for a dependency on `tokio`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to send request: The server name or address could not be resolved
; class=Os (2)

[kamyuentse]

Same here.

• Network Topology: local ---> cntlm ---> Company Proxy Server
• The $HOME/.cargo/config configured to use the cntlm proxy

I use Wireshark to monitor the connection between local and cntlm. I notice local(cargo?) reply an Encrypted Alert packet followed by a RST from cntlm.