fluvio
fluvio copied to clipboard
Tracking issue: Support running a cluster in WSL2
Support Kubernetes/ WSL2:
We do not yet support the Windows Subsystem for Linux (WSL), but we have an interest in making at least the client crate and the CLI work nicely in that environment. In this issue I want to capture some of the current pain points that users have experienced in WSL.
@bbarekas
on Discord seems to be having trouble with the minikube setup on WSL. The problem seems to be that the miniube tunnel would not come up and so the SC could not create an SPU.
- Error output:
❯ sudo true && (sudo nohup minikube tunnel >/tmp/tunnel.out 2>/tmp/tunnel.out &)
❯ cat /tmp/tunnel.out
* Profile "minikube" not found. Run "minikube profile list" to view all profiles.
To start a cluster, run: "minikube start"
❯ fluvio cluster start
✅ ok: Kubernetes config is loadable
✅ ok: Supported helm version is installed
✅ ok: Fixed: Minikube tunnel not found
✅ ok: Fluvio system charts are installed
💙 note: Fluvio is already running
Error:
0: Fluvio cluster error
1: Fluvio cluster error
2: Failed to install Fluvio on Kubernetes
3: Timed out when waiting for SC service
❯ cat /tmp/tunnel.out
I0429 16:09:57.893138 5508 out.go:278] Setting OutFile to fd 1 ...
log: exiting because of error: log: unable to create log: open /tmp/minikube_tunnel_9def94cfbf18906139a3866da01f4abff40d81de_0.log: permission denied
Notably, the minikube tunnel
command seemed to be printing an error that we haven't seen before. This was run after minikube start
, when minikube was known to be running.
~ ❯ sudo minikube tunnel
🤷 Profile "minikube" not found. Run "minikube profile list" to view all profiles.
👉 To start a cluster, run: "minikube start"
In minikube , we are able to see the SC pod but not SPU, and we see the sc-public
service as <pending>
. Both of these are usually caused by problems with the minikube tunnel.
❯ kubectl get all -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/fluvio-sc-76b59f697-2hxvs 1/1 Running 0 86m
kube-system pod/coredns-74ff55c5b-vq29d 1/1 Running 0 117m
kube-system pod/etcd-minikube 1/1 Running 0 117m
kube-system pod/kube-apiserver-minikube 1/1 Running 0 117m
kube-system pod/kube-controller-manager-minikube 1/1 Running 0 117m
kube-system pod/kube-proxy-8qd5n 1/1 Running 0 117m
kube-system pod/kube-scheduler-minikube 1/1 Running 0 117m
kube-system pod/storage-provisioner 1/1 Running 0 117m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/fluvio-sc-internal ClusterIP 10.103.122.61 <none> 9004/TCP 86m
default service/fluvio-sc-public LoadBalancer 10.99.192.28 <pending> 9003:32533/TCP 86m
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 117m
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 117m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR
AGE
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 117m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default deployment.apps/fluvio-sc 1/1 1 1 86m
kube-system deployment.apps/coredns 1/1 1 1 117m
NAMESPACE NAME DESIRED CURRENT READY AGE
default replicaset.apps/fluvio-sc-76b59f697 1 1 1 86m
kube-system replicaset.apps/coredns-74ff55c5b 1 1 1 117m
- System details:
~ ❯ uname -a
Linux sputnik 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
~ ❯ fluvio version
Fluvio CLI : 0.7.3
Fluvio CLI SHA256 : 6ff4969b99a67ee9d8f594475e1884f51313dd99ac5414c4a13ab797845a590f
Fluvio Platform : 0.7.3 (cloud)
Git Commit : 895c07f66347d089c2b85a9df28ed1a4b71f3cb3
OS Details : Ubuntu 20.04 (kernel 5.4.72-microsoft-standard-WSL2)
=== Plugin Versions ===
Fluvio Runner (fluvio-run) : 0.1.0
Fluvio Cloud CLI (fluvio-cloud) : 0.1.5
~ ❯ minikube version
minikube version: v1.19.0
commit: 15cede53bdc5fe242228853e737333b09d4336b5
~ ❯ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Here is kubectl config view
output:
~ ❯ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/bbarekas/.minikube/ca.crt
extensions:
- extension:
last-update: Thu, 29 Apr 2021 18:36:14 EEST
provider: minikube.sigs.k8s.io
version: v1.19.0
name: cluster_info
server: https://127.0.0.1:60655
name: minikube
contexts:
- context:
cluster: minikube
extensions:
- extension:
last-update: Thu, 29 Apr 2021 18:36:14 EEST
provider: minikube.sigs.k8s.io
version: v1.19.0
name: context_info
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /home/bbarekas/.minikube/profiles/minikube/client.crt
client-key: /home/bbarekas/.minikube/profiles/minikube/client.key
~ ❯
Investigating whether solving https://github.com/infinyon/fluvio/issues/668 will solve this issue
For this release, we support client support only
Stale issue message
Using k3d
on wsl2
(on Windows 11), I had no problems getting started:
❯ fluvio cluster start
Current channel: stable
📝 Running pre-flight checks
✅ Kubectl active cluster k3d-mycluster at: https://0.0.0.0:44621 found
✅ Supported helm version 3.9.4+gdbc6d8e is installed
✅ Supported Kubernetes server 1.23.8+k3s1 found
✅ Fixed: Fluvio Sys chart 0.9.33 is installed
✅ Previous fluvio installation not found
🎉 All checks passed!
✅ Installed Fluvio app chart: 0.9.33
✅ Connected to SC: 172.18.0.2:30003
👤 Profile set
✅ SPU group main launched with 1 replicas
🎯 Successfully installed Fluvio!
~ took 5m19s
❯ fluvio version
Release Channel : stable
Fluvio CLI : 0.9.33
Fluvio CLI SHA256 : 6a1bebc4afea1029c86f1ec49a85a9f3da98791e4858562b92813cfddc72b4b5
Fluvio channel frontend SHA256 : 3742cb7da3722b6f9ec2d0534d587a29cbb877f574f2b8e76d91c0db0a78998b
Fluvio Platform : 0.9.33 (k3d-mycluster)
Git Commit : d63e3e2569e4d64a098e5c2189ac68e6e9cd2670
OS Details : Ubuntu 22.04 (kernel 5.10.60.1-microsoft-standard-WSL2)
=== Plugin Versions ===
Fluvio Runner (fluvio-run) : 0.0.0
Infinyon Cloud CLI (fluvio-cloud) : 0.1.8
I have not tried with minikube
or kind
.
Have you tried performing simple produce and consume? Unfortunately, Github Action doesn't support WSL2 yet: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md, so we can't put in CI yet.
maybe this: https://github.com/actions/runner-images/issues/5760#issuecomment-1157585475
@sehz Yep, producing/consuming worked fine. WSL2 uses a real Linux kernel, so in theory it should behave mostly identically to the same distro (Ubuntu by default) running directly on the hardware.
If nothing else, I think it's safe to say that this issue is specific to Minikube.
Since this works on WSL2 with K3d, we will mark it as close for now. Thanks @BatmanAoD