kitops-ml
Add option to generate an OCI container image
More and more tools and features arise that support models as OCI compliant artifacts.
- Kubernetes will allow mounting container images as volumes from Version 1.31 onwards which is more elegant than InitContainers
- KServe supports "ModelCars" which is similar. An inference service, based on a model in a container image, will be deployed.
Unfortunately, ModelKits cannot be natively used with those tools/features. Kubernetes throws Failed to pull image ...: parsing image configuration: unsupported image-specific operation on artifact with type "application/vnd.kitops.modelkit.config.v1+json" and KServe ModelCar-Images need a shell to function.
We are using Red Hat OpenShift AI which introduces KServe and ModelCars as well. Therefore, we would like to use KitOps as a tool to build those "ModelCars". Otherwise, building an OCI image out of an OCI artifact (ModelKit) by introducing another tool or process seems to me a little bit too complex.
An option to package the ModelKit into an OCI Image, which is based on a micro image that the user can choose could be a great option to make KitOps more interoperable.
@notniknot Have you checked https://github.com/jozu-ai/kitops/tree/main/build/dockerfiles/KServe to see if this would provide the integration that you need with KServe.
We are also working under the CNCF umbrella with other organizations to create a standard similar to ModelKits that will allow the whole CNCF to get behind.
There is also the kitops-init container (documentation here)
On the other hand, it would not be terribly difficult to convert a ModelKit into a OCI container, so this is something we could pursue.
Thanks @gorkem and @amisevsk for the information! I did not know about ClusterStorageContainers.
But unfortunately, as far as I can tell, Red Hat OpenShift AI only supports paths to either S3 storage or an OCI image (see this blog post as example https://developers.redhat.com/articles/2025/01/30/build-and-deploy-modelcar-container-openshift-ai#deploy_a_model_server_with_a_modelcar_container). If it wouldn't be too difficult, being able to convert a ModelKit to an OCI Container with customizable base image would still be really amazing.
@notniknot We have also implemented a way to convert ModelKits to containers that can work with any system with Jozu Hub. It is a feature that is called Jozu Hub Containers and you can read about how it works on the docs . It is not part of KitOps because it needed to be a server. It is another alternate solution that popped to my mind.
Yes, you are right @gorkem, this is a feature that could work for Jozu's enterprise users. As far as I remember, Jozu adds the ModelKit layers to another base image on request?
Anyways, I believe that enabling users to build an OCI image from the start would give KitOps users a big benefit. Do you think this is out of scope because this problem could be solved in a different way via Jozu Hub?
Yes, you are right @gorkem, this is a feature that could work for Jozu's enterprise users. As far as I remember, Jozu adds the ModelKit layers to another base image on request?
Correct, that is roughly how it works. Jozu Hub maintains the runtime stack and adds models on demand.
Anyways, I believe that enabling users to build an OCI image from the start would give KitOps users a big benefit. Do you think this is out of scope because this problem could be solved in a different way via Jozu Hub?
It is not out of scope. Needs the right experience.
Let me know if you need someone to test it in case you want to implement that feature :)