Network requests fails in Windows with default k3d

[deepu105]

In windows the library fails with 'HyperError: error trying to connect: TCP connect error: The requested address is not valid in its context. (os error 10049)'

Here is a reproducible code. I tried with both Docker-Desktop Kubernetes and K3D and both fails. I'll test with a cloud-based cluster to see if that works

maybe this could be due to how docker works in Windows and maybe we can do what the go lib does on that situation

use k8s_openapi::api::core::v1::Node;
use kube::{api::ListParams, Api, Error, Resource};

#[tokio::main]
async fn main() -> core::result::Result<(), Error> {
    let client = kube::Client::try_default().await.unwrap();

    let lp = ListParams::default();
    let api_nodes: Api<Node> = Api::all(client.clone());

    match api_nodes.list(&lp).await {
        Ok(node_list) => {
            node_list
                .iter()
                .for_each(|node| println!("{}", node.name()));
        }
        Err(e) => {
            panic!("{}", e)
        }
    }
    Ok(())
}

[deepu105]

I would love to contribute. If you can give some tips on where to start I can try to look into this

[MikailBag]

I think starting point is to create a simple request (such as GET /version) and to submit it using

a) kube::service::Service b) manually created hyper client

Based on results you should be able to tell who is at fault: kube, hyper or something else.

[deepu105]

So I just tried with a GKE cluster and it works so the issue is definitely related to local Kubernetes clusters via docker. And since Docker doesn't expose localhost in Windows like in Unix it could be the issue. I'll look at the Go client on what it does.

@MikailBag thanks, i'll try that

[deepu105]

Ok seems like this is the issue. For the k3d cluster created the server URL is https://0.0.0.0:63376 since it's on Docker. If I change the URL to https://127.0.0.1:63376 then the connection works. @clux @MikailBag do you think we can do something in this case?

[clux]

Hm, this might be a case where we tell people to use k3d cluster create more accurately:

From docs on k3d cluster create:

      --api-port [HOST:]HOSTPORT                                       Specify the Kubernetes API server port exposed on the LoadBalancer (Format: [HOST:]HOSTPORT)
                                                                        - Example: `k3d cluster create --servers 3 --api-port 0.0.0.0:6550`

setting --api-port 127.0.0.1:6550 might be a solution? Possibly similar issue: https://github.com/rancher/k3d/issues/423

Strange that 0.0.0.0 doesn't work, as I thought that meant to listen on all addresses, but only had time to skim here.

[deepu105]

Ya but the problem is k3d's quick start doesnt mention needing to use this so people, like me, will end up just running k3d cluster create and end up with this issue. But the Go client seems to work for both 0.0.0.0 and 127.0.0.1 on Windows so they are doing something to make it work I guess

[clux]

Yeah, that's true. Maybe if you can find what client-go is doing here, we can potentially duplicate it?

[deepu105]

I'll try to look into it. But I have never looked at the go client before so gonna take some time

[deepu105]

From my initial look around it doesnt seem like Go client does anything special for this. I might have missed something. Could this be a Hyper issue?

[clux]

I had a couple of searches through hyper and only couple of things I can think of are things in the connector. Maybe new_with_resolver or set_local_addresses will do helpful things here? Not very familiar in this area.

This might be easier to pinpoint now that we can create custom clients, perhaps it would be helpful experimenting a bit with how various hyper options interact on windows. We can customise the hyper::Client or its Connector, and use minimal layers via https://github.com/clux/kube-rs/blob/0339b85318d295609c4d69f9753ad832b8ba10d2/examples/custom_client.rs#L17-L23