Tracking issue: Support running a cluster in WSL2

[nicholastmosher]

Support Kubernetes/ WSL2:

We do not yet support the Windows Subsystem for Linux (WSL), but we have an interest in making at least the client crate and the CLI work nicely in that environment. In this issue I want to capture some of the current pain points that users have experienced in WSL.

---

@bbarekas on Discord seems to be having trouble with the minikube setup on WSL. The problem seems to be that the miniube tunnel would not come up and so the SC could not create an SPU.

• Error output:

❯ sudo true && (sudo nohup minikube tunnel >/tmp/tunnel.out 2>/tmp/tunnel.out &)
❯ cat /tmp/tunnel.out
* Profile "minikube" not found. Run "minikube profile list" to view all profiles.
  To start a cluster, run: "minikube start"
❯ fluvio cluster start
✅ ok: Kubernetes config is loadable
✅ ok: Supported helm version is installed
✅ ok: Fixed: Minikube tunnel not found
✅ ok: Fluvio system charts are installed
💙 note: Fluvio is already running
Error:
   0: Fluvio cluster error
   1: Fluvio cluster error
   2: Failed to install Fluvio on Kubernetes
   3: Timed out when waiting for SC service
❯ cat /tmp/tunnel.out
I0429 16:09:57.893138    5508 out.go:278] Setting OutFile to fd 1 ...
log: exiting because of error: log: unable to create log: open /tmp/minikube_tunnel_9def94cfbf18906139a3866da01f4abff40d81de_0.log: permission denied

Notably, the minikube tunnel command seemed to be printing an error that we haven't seen before. This was run after minikube start, when minikube was known to be running.

~ ❯ sudo minikube tunnel
🤷  Profile "minikube" not found. Run "minikube profile list" to view all profiles.
👉  To start a cluster, run: "minikube start"

In minikube , we are able to see the SC pod but not SPU, and we see the sc-public service as <pending>. Both of these are usually caused by problems with the minikube tunnel.

❯ kubectl get all -A
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE
default       pod/fluvio-sc-76b59f697-2hxvs          1/1     Running   0          86m
kube-system   pod/coredns-74ff55c5b-vq29d            1/1     Running   0          117m
kube-system   pod/etcd-minikube                      1/1     Running   0          117m
kube-system   pod/kube-apiserver-minikube            1/1     Running   0          117m
kube-system   pod/kube-controller-manager-minikube   1/1     Running   0          117m
kube-system   pod/kube-proxy-8qd5n                   1/1     Running   0          117m
kube-system   pod/kube-scheduler-minikube            1/1     Running   0          117m
kube-system   pod/storage-provisioner                1/1     Running   0          117m

NAMESPACE     NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
default       service/fluvio-sc-internal   ClusterIP      10.103.122.61   <none>        9004/TCP                 86m
default       service/fluvio-sc-public     LoadBalancer   10.99.192.28    <pending>     9003:32533/TCP           86m
default       service/kubernetes           ClusterIP      10.96.0.1       <none>        443/TCP                  117m
kube-system   service/kube-dns             ClusterIP      10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP   117m

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR
AGE
kube-system   daemonset.apps/kube-proxy   1         1         1       1            1           kubernetes.io/os=linux   117m

NAMESPACE     NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
default       deployment.apps/fluvio-sc   1/1     1            1           86m
kube-system   deployment.apps/coredns     1/1     1            1           117m

NAMESPACE     NAME                                  DESIRED   CURRENT   READY   AGE
default       replicaset.apps/fluvio-sc-76b59f697   1         1         1       86m
kube-system   replicaset.apps/coredns-74ff55c5b     1         1         1       117m

• System details:

~ ❯ uname -a
Linux sputnik 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

~ ❯ fluvio version
Fluvio CLI           : 0.7.3
Fluvio CLI SHA256    : 6ff4969b99a67ee9d8f594475e1884f51313dd99ac5414c4a13ab797845a590f
Fluvio Platform      : 0.7.3 (cloud)
Git Commit           : 895c07f66347d089c2b85a9df28ed1a4b71f3cb3
OS Details           : Ubuntu 20.04 (kernel 5.4.72-microsoft-standard-WSL2)
=== Plugin Versions ===
Fluvio Runner (fluvio-run)     : 0.1.0
Fluvio Cloud CLI (fluvio-cloud) : 0.1.5

~ ❯ minikube version
minikube version: v1.19.0
commit: 15cede53bdc5fe242228853e737333b09d4336b5

~ ❯ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

[bbarekas]

Here is kubectl config view output:

~ ❯ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /home/bbarekas/.minikube/ca.crt
    extensions:
    - extension:
        last-update: Thu, 29 Apr 2021 18:36:14 EEST
        provider: minikube.sigs.k8s.io
        version: v1.19.0
      name: cluster_info
    server: https://127.0.0.1:60655
  name: minikube
contexts:
- context:
    cluster: minikube
    extensions:
    - extension:
        last-update: Thu, 29 Apr 2021 18:36:14 EEST
        provider: minikube.sigs.k8s.io
        version: v1.19.0
      name: context_info
    namespace: default
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /home/bbarekas/.minikube/profiles/minikube/client.crt
    client-key: /home/bbarekas/.minikube/profiles/minikube/client.key
~ ❯

[nicholastmosher]

Investigating whether solving https://github.com/infinyon/fluvio/issues/668 will solve this issue

[sehz]

For this release, we support client support only

[github-actions[bot]]

Stale issue message

[BatmanAoD]

Using k3d on wsl2 (on Windows 11), I had no problems getting started:

❯ fluvio cluster start
Current channel: stable
📝 Running pre-flight checks
    ✅ Kubectl active cluster k3d-mycluster at: https://0.0.0.0:44621 found
    ✅ Supported helm version 3.9.4+gdbc6d8e is installed
    ✅ Supported Kubernetes server 1.23.8+k3s1 found
    ✅ Fixed: Fluvio Sys chart 0.9.33 is installed
    ✅ Previous fluvio installation not found
🎉 All checks passed!
✅ Installed Fluvio app chart: 0.9.33
✅ Connected to SC: 172.18.0.2:30003
👤 Profile set
✅ SPU group main launched with 1 replicas
🎯 Successfully installed Fluvio!

~ took 5m19s
❯ fluvio version
Release Channel      : stable
Fluvio CLI           : 0.9.33
Fluvio CLI SHA256    : 6a1bebc4afea1029c86f1ec49a85a9f3da98791e4858562b92813cfddc72b4b5
Fluvio channel frontend SHA256 : 3742cb7da3722b6f9ec2d0534d587a29cbb877f574f2b8e76d91c0db0a78998b
Fluvio Platform      : 0.9.33 (k3d-mycluster)
Git Commit           : d63e3e2569e4d64a098e5c2189ac68e6e9cd2670
OS Details           : Ubuntu 22.04 (kernel 5.10.60.1-microsoft-standard-WSL2)
=== Plugin Versions ===
Fluvio Runner (fluvio-run)     : 0.0.0
Infinyon Cloud CLI (fluvio-cloud) : 0.1.8

I have not tried with minikube or kind.

[sehz]

Have you tried performing simple produce and consume? Unfortunately, Github Action doesn't support WSL2 yet: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md, so we can't put in CI yet.

[sehz]

maybe this: https://github.com/actions/runner-images/issues/5760#issuecomment-1157585475

[BatmanAoD]

@sehz Yep, producing/consuming worked fine. WSL2 uses a real Linux kernel, so in theory it should behave mostly identically to the same distro (Ubuntu by default) running directly on the hardware.

If nothing else, I think it's safe to say that this issue is specific to Minikube.

[sehz]

Since this works on WSL2 with K3d, we will mark it as close for now. Thanks @BatmanAoD