k3s
k3s copied to clipboard
k3s-io
windows support
there are pretty much customers are using windows at edge side device with installation of labview.
This one is tricky. I personally don't know much about Kubernetes on Windows, but Rancher/RKE works on windows so I can find out what needs to be done here. If there's demand, I'd love to do this.
on windows in general it is using a vm hyperv or vbox to run kubenetes like minikube doing. But minikube is really not production ready and there is a lot of issue and no any customization capability.
Looking at Microsoft's instructions for joining a Windows node to a cluster, Kubernetes releases have binaries for running Windows worker nodes (kubelet.exe, kube-proxy.exe), but not the control plane. I see there's also an official Windows binary for Flannel, which Microsoft's PS scripts leverage.
I would think a K3s Windows build with only the agent components would be possible. At this point, it looks like Containerd support in Windows is still only in alpha and the official Windows build is still a WIP. Not sure what other dependencies in K3s would be problematic for an agent-only Windows build.
@ibuildthecloud
Hey guys, I just checked the Containerd support for Windows and sounds like it's no longer in alpha state. I am using K3S two of my side projects and supporting Windows on the edge is one of my highest priorities in both projects.
We also don't really need to run the control plane on the edge and Agent support on the Window should be more than enough. I do not know enough about the implementation details of the K3S and K8S, but I am more than happy to help to get the Windows support done.
Is there any progress on this ticket since the last comment posted on 11 April?
my fork can start and join a windows master node. It does appear to have done a few things right, example my kubelet lists container images on my node (I am using docker desktop community). However I don't have networking setup, so the node stays unschedulable.
I would love for someone with more understanding of network configuration to take it further. And give instructions on how to setup networking. @ibuildthecloud do you know anyone that could help here?
cmd I used to get it to "somewhat" startup
.\k3s.exe server -d c:\tmp\k3s --flannel-backend none --docker --kube-proxy-arg "proxy-mode=userspace" --disable-network-policy
Branch I was using for this experiment. https://github.com/KnicKnic/k3s/tree/k3s_run_windows_node
There are few errors (these related to cni / networking)
W0223 00:59:39.172343 20244 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
E0223 00:59:39.937342 20244 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
And this random one for TLS, have no clue why a cert would be wrong.
2020-02-23 00:54:37.591274 I | http: TLS handshake error from 127.0.0.1:2111: remote error: tls: bad certificate
2020-02-23 00:54:37.602363 I | http: TLS handshake error from 127.0.0.1:2114: remote error: tls: bad certificate
@thxCode @alena1108 @cloudnautique @aiwantaozi pinging some other people in rancher that seemed to have worked on windows integration (for rancher) to see if they could help with setting up windows networking. See my previous comment https://github.com/rancher/k3s/issues/114#issuecomment-590047668 .
@KnicKnic I just bookmarked these:
https://github.com/microsoft/SDN/blob/master/Kubernetes/flannel/register-svc.ps1 a bunch of script for establishing a SDN on Windows.
In particular https://github.com/microsoft/SDN/search?q=cniconfig&unscoped_q=cniconfig Powershell scripts that generate cni configuration.
This is old How To Guide: On-premise Kubernetes for Windows using Flannel (Host-Gateway)
Troubleshooting Kubernetes Networking on Windows Part 1
Also https://github.com/microsoft/SDN/issues/170#issuecomment-548574978
Even though I can't run k3s agent on Windows right now, it should be possible to connect a k8s node on Windows to a k3s control node on Linux, yes?
@jiridanek vanilla k8s kubelets cannot connect to k3s servers, as they are missing the k3s-specific bootstrap code.
@jiridanek vanilla k8s kubelets cannot connect to k3s servers, as they are missing the k3s-specific bootstrap code.
Any way to work around that? Perhaps a local adapter of some sort?
Very interested in a k3s windows agent as well, has there been any updates here? Happy to contribute here as well
@ibuildthecloud Is there some progress or status on the Windows K3s Agent? Is there some Branch or something to work on to speed things up?
Thanks
+1 would be keen for this (windows agent + linux server), what is the "k3s-specific bootstrap code" ? (if there's an arch doc, I can look at that)
+1 I too would like to have a package to add in a windows host as a worker node! :)
@moley +1 We'd like to use k3s in a windows environment too to handle linux pods
Windows nodes will run Windows containers only. This is true Kubernetes on Windows, not Kubernetes in WSL or anything like that.
Yes, that's what I need. I've some Linux VM for running k3s and want to join some windows VM for running workload which only works on Windows.
@moley +1 We'd like to use k3s in a windows environment too to handle linux pods
Windows nodes will run Windows containers only. This is true Kubernetes on Windows, not Kubernetes in WSL or anything like that.
Ok, so we will need a linux vm in our windows environment to get this running? Are there plans to enable this directly in the future?
Now with Docker Desktop being a paid solution for enterprises this would be valuable for Rancher Desktop.
Rancher Desktop already runs K3s on Windows without any additional work necessary, albeit in a sandbox.
Hello all. I just wanted to check. Has anyone been able to get a windows worker node to join a k3s cluster?
In Nomad (https://www.nomadproject.io) the network is normally working on a plugin https://github.com/containernetworking/plugins
We do not currently have any plans to add native Windows support to K3s.
@brandond , pleeeeeease?
Hello. Maybe a different question to ask. But would it be possible to manually add a windows node to a K3s cluster? Has anyone tried? If so, did you encounter any issues?
Hello. Maybe a different question to ask. But would it be possible to manually add a windows node to a K3s cluster? Has anyone tried? If so, did you encounter any issues?
You could probably experiment with rancher desktop on Windows BUT it will only run Linux containers via WSL2 so it is probably not what everyone is wishing for - a windows container runtime inside a cluster.
