bundle-kubeflow icon indicating copy to clipboard operation
bundle-kubeflow copied to clipboard

Modify the NIMs how to guide after upgrading to KServe 0.14

Open NohaIhab opened this issue 4 months ago • 1 comments

Context

The Deploy NVIDIA NIM on Charmed Kubeflow how to guide is currently deviating from the NVIDIA’s runtime YAMLs by setting the NIM_CACHE_PATH to /tmp. This is to avoid hitting the issue https://github.com/kserve/kserve/issues/3687 in KServe with downloading models into PVCs. However, this issue is expected to be fixed in KServe version 0.14 and Charmed Kubeflow 1.10 will be supporting KServe 0.14. Once we bump the KServe version in CKF to 0.14 i.e. release CKF 1.10, we should update this how to guide to use the NVIDIA runtime YAMLs, where PVs and PVCs are being utilized.

What needs to get done

Update the runtime in the how to guide to use PVCs

Definition of Done

NIMs how to guide is updated and approved

NohaIhab avatar Oct 18 '24 13:10 NohaIhab