cncf
[Initiative]: Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers
Name
Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers
Short description
Integrating the AI developer inner loop into an end-to-end CI/CD process leveraging cloud-native technologies and tooling
Responsible group
TOC
Does the initiative belong to a subproject?
Yes
Subproject name
TOC Artificial Intelligence Initiatives
Primary contact
Vincent Caldeira ([email protected])
Additional contacts
Ricardo Aravena ([email protected])
Initiative description
Scope definition
Focus on the developer inner loop, everything an AI engineer does on a laptop/desktop before code or models ever reach CI/CD in a cloud-native environment:
- Local container workspaces: Reference inner loop workflow using desktop tooling such as Podman Desktop / Podman AI Lab for root-less, GPU-aware experimentation, including template images for PyTorch/LLM stacks and volume-mounted datasets.
- Unified model build & run CLI: Hardening inference on developer machine and agentic frameworks to leverage container-based tooling so engineers can easily spin-up inference, RAG and multi-agent services locally with one command.
- Standard packaging of artefacts: Drive convergence between various implementations such as ModelKit, ModelCar towards the emerging ModelPack spec to create a single OCI-manifest that can hold model weights, metadata and SBOM.
- Inner-loop supply-chain security: Integrate Notary v2 / model authenticity and transparency via Sigstore, LF AI & Data Model Openness Framework-generated model & data cards, plus SBOM annotations directly into the OCI artefact so that security & openness are “baked in” before CI.
- Fast hand-off to outer loop: Provide reference GitOps flows (Flux/Argo) that pull the signed artefact into KServe with ModelPack image-mount optimisation, and register versions in Kubeflow Model Registry.
Why it matters for the CNCF
- Closes the skills gap: Today AI engineers live in Python notebooks while cloud-native tools live in YAML. A container-native inner loop brings AI creators into the CNCF ecosystem early, making Kubernetes the default target platform.
- Eliminates fragmentation: Multiple, incompatible model-packaging attempts (Docker model-CLI, KitOps, ONNX zip files, etc.) slow adoption. A CNCF-backed, OCI-compatible spec creates a neutral home and clear interoperability story.
- Raises baseline security & transparency: By embedding MOF openness requirements and Sigstore signing before code hits CI, the sub-stream aligns with industry compliance trends and improves trust across end-to-end supply chains.
- Accelerates project reuse: The work provides reusable libraries, CRDs and GitOps templates that every other CNCF AI project (KServe, Kubeflow, TrustyAI, etc.) can import rather than reinventing developer tooling.
Key technologies & projects involved
- Container tooling: Podman Desktop, Podman AI Lab, Docker model-runner (observer role)
- Packaging & spec: ModelKit (KitOps), ModelPack, ModelCar, OCI image/artefact spec, Notary v2, Sigstore
- Model runtime & APIs: Ramalama (potential contribution), Agentic Orchestration Frameworks, MCP Servers for tool orchestration
- Kubernetes services: KServe + ModelCars, Kubeflow Model Registry
- Governance & openness: LF AI & DATA Model Openness Framework (MOF) generators, SBOM annotations
- GitOps & automation: Flux, Argo Workflows/Pipelines
Deliverable(s) or exit criteria
- An technical POC showing <10 min “idea-to-inference” path for cloud-native agent development on a developer laptop.
- Clearly documented standards for OCI artefact standardization across runtimes and registries.
- Specification / procedure to achieve MOF Class III compliant model distributions via any OCI registry.
- Standardised process for leveraging model signing with artefacts-level provenance to support a verified end-to-end CI/CD reference pipeline including outer loop for AI engineering.
Interesting proposal. I like the 'bottom-up' focus.. starting with the ai engineer.
@caldeirav agree w/ bottom up approach.
Please outline the project timeline and break up the deliverables if necessary.
Okay to move to vote @riaankleinhans
This initiative has been approved by the TOC and is ready to be worked on with the appropriate TAG and TOC liaison.
@caldeirav Feel free to also include me for local LLM initiatives; I have some experience working with developers for local inference tooling.
