Optimize K8S requests to don't bulk the K8S API

[tty47]

hello!

currently, we are making a request per resource, so, at scale, this might be an issue, let's try to find a better way to do it some ideas to avoid doing it in this way:

1. Adjust QPS (Queries Per Second) and Burst Settings The Kubernetes client-go library allows you to configure the QPS (Queries Per Second) and burst settings for API requests. By default, these settings might be too low for your application’s needs. You can increase them to reduce throttling. Here's an example of how to adjust these settings:

import (
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
)

func main() {
	config, err := rest.InClusterConfig()
	if err != nil {
		panic(err.Error())
	}

	// Increase QPS and Burst settings
	config.QPS = 100
	config.Burst = 200

	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		panic(err.Error())
	}

	// Use clientset to interact with the Kubernetes API
}

Adjust the values of QPS and Burst according to your application's requirements and the capacity of your Kubernetes API server.

2. Optimize API Calls

• Review your application logic to see if you can reduce the number of API calls. Some strategies include:
• Batching requests: Combine multiple operations into a single request if possible.
• Caching responses: Cache responses locally to avoid making the same request multiple times.
• Reduce polling frequency: If you are polling the API, consider reducing the frequency of these requests.

3. Aggregating API Requests While Kubernetes API does not natively support batch operations in a single HTTP request, you can aggregate requests at the application level. This means collecting multiple operations and executing them sequentially or in parallel within the application but with fewer API interactions. Example: Aggregating Service Creation Suppose you need to create multiple Kubernetes services. Instead of making individual API calls, you can batch the service creation in your application code.

import (
	"context"
	"fmt"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	corev1 "k8s.io/api/core/v1"
)

func main() {
	config, err := rest.InClusterConfig()
	if err != nil {
		panic(err.Error())
	}

	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		panic(err.Error())
	}

	services := []corev1.Service{
		{
			ObjectMeta: metav1.ObjectMeta{
				Name: "service1",
			},
			Spec: corev1.ServiceSpec{
				Ports: []corev1.ServicePort{
					{
						Port: 80,
					},
				},
				Selector: map[string]string{
					"app": "app1",
				},
			},
		},
		{
			ObjectMeta: metav1.ObjectMeta{
				Name: "service2",
			},
			Spec: corev1.ServiceSpec{
				Ports: []corev1.ServicePort{
					{
						Port: 80,
					},
				},
				Selector: map[string]string{
					"app": "app2",
				},
			},
		},
	}

	createServices(clientset, "default", services)
}

func createServices(clientset *kubernetes.Clientset, namespace string, services []corev1.Service) {
	for _, service := range services {
		_, err := clientset.CoreV1().Services(namespace).Create(context.TODO(), &service, metav1.CreateOptions{})
		if err != nil {
			fmt.Printf("Error creating service %s: %v\n", service.Name, err)
		} else {
			fmt.Printf("Service %s created successfully\n", service.Name)
		}
	}
}

cc: @smuu @mojtaba-esk

[smuu]

One idea to achieve that is to introduce a new function like BulkStart, which takes a list of Instances and creates 1 API request instead of one for each Instance.

[mojtaba-esk]

I am adding this for the record. I found something useful and interesting to work on. Fortunately, the Kubernetes (k8s) SDK defines interfaces for many things, like Pods. We can use these to create our own custom interface that extends the k8s SDK, allowing us to add any API we want. For example, we can add something like this:

type ExtendedPodInterface interface {
    PodInterface
    CreateBatch(ctx context.Context, pods []*v1.Pod, opts metav1.CreateOptions) ([]*v1.Pod, error)
}

This lets us define additional methods, such as creating multiple Pods at once.

[tty47]

update:

we've found some limitations in the Kubernetes API that block us from tweaking the client config, the Kubernetes API doesn't recognize a list of resources and requires us to make a single request per resource.

the API docs: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#create-replicaset-v1-apps even in the docs the Replicasets is in plural, it only accepts one: create a ReplicaSet

[tty47]

closed based on the latest message