Nobuto Murata comments

Results 47 comments of


                                            Nobuto Murata

enable gpu fails on 20.04 (focal) and MicroK8s 1.21 - failed to get sandbox runtime: no runtime for "nvidia" is configured

I think a previously opened issue in https://github.com/canonical/microk8s/issues/2575 might have the same root cause, but at that point a solution was to use 1.22, which doesn't support Kubeflow in this...

enable gpu fails on 20.04 (focal) and MicroK8s 1.21 - failed to get sandbox runtime: no runtime for "nvidia" is configured

Ah, okay. You are talking about [this change](https://github.com/canonical/microk8s/commit/1b55a80ba01659dc46524436dfef462d1e4208fa#diff-56759910381a014fecfd7556dd72ddd68c747d922a5b7df2044b9ce7c552f5f5).

enable gpu fails on 20.04 (focal) and MicroK8s 1.21 - failed to get sandbox runtime: no runtime for "nvidia" is configured

Okay, at least the theory has been confirmed; the patchset below can make GPU enablement step work. https://github.com/canonical/microk8s/compare/1.21...nobuto-m:1.21-gpu

enable gpu fails on 20.04 (focal) and MicroK8s 1.21 - failed to get sandbox runtime: no runtime for "nvidia" is configured

> > Okay, at least the theory has been confirmed; the patchset below can make GPU enablement step work. [1.21...nobuto-m:1.21-gpu](https://github.com/canonical/microk8s/compare/1.21...nobuto-m:1.21-gpu) > > @nobuto-m , Can you pls help me how...

AutoML jobs times out in pipeline until katib-controller is restarted

> I tried to reproduce your issue but in my case the trials get completed without patching the katib deployment: Hmm, interesting. I had the timeout 100% so far. >...

AutoML jobs times out in pipeline until katib-controller is restarted

Hmm, `trial-resources=Workflow.v1alpha1.argoproj.io` might be a red herring. The issue is still reproducible in my environment, but after adding `trial-resources=Workflow.v1alpha1.argoproj.io` and removing it again, the trial jobs complete. So the key...