Leverage Kubernetes OCI image as volume feature

[ritazh]

Is your feature request related to a problem? Please describe.

Reduce latency for pulling model weights

Describe the solution you'd like

Leverage the ImageVolume feature in Kubernetes to allow model weights to be mounted to a pod as an OCI image.

Describe alternatives you've considered

Additional context

[zhuangqh]

let us do some test to see how much time we can benefit from using oci-volume. @nojnhuh phi4 model is suitable for this test. Use this script to generate a testing manifest. https://github.com/kaito-project/kaito/blob/main/presets/workspace/test/scripts/README.md

[sdesai345]

@zhuangqh moving this item to In Progress

[sdesai345]

@zhuangqh can you provide an update here?