ramalama icon indicating copy to clipboard operation
ramalama copied to clipboard

Add support for kserve

Open rhatdan opened this issue 8 months ago • 1 comments
trafficstars

Summary by Sourcery

This pull request introduces support for generating KServe configurations, allowing users to deploy AI models on Kubernetes using the KServe framework. It adds a new kserve option to the ramalama serve --generate command, which generates the necessary YAML files for deploying a model as a KServe service.

New Features:

  • Adds support for generating KServe YAML definitions for running AI models as a service in Kubernetes, enabling deployment and management of models using the KServe framework.

Enhancements:

  • The ramalama serve command now accepts a --generate kserve option to generate KServe YAML files.
  • The generated KServe YAML files include resource requests and limits for CPU, memory, and GPU (if available).

rhatdan avatar Feb 24 '25 18:02 rhatdan