Define a benchmark for performances

[bsctl]

We should define a benchmark for performances and tests in different stress conditions.

For example:

• how many tenants
• how many namespaces/tenants
• how many concurrent users
• how many requests

[prometherion]

Could you elaborate a bit more? Because, at first glance, I'd say it's a matter of Kubernetes limits, here :)

Especially for:

• tenants
• namespaces/tenants
• concurrent users

Regarding requests, I'd say it depends on the resources assigned to Capsule, since we got the webhook server there.

[bsctl]

We have to provide a benchmark to the adopters. If most of limits are matter of Kubernetes, let's just to document it. It would be nice to have a performance test suite, not part of this code base.

[prometherion]

I'd say #150 could help here: my plan is to scrape metrics and perform some massive fuzz activities in order to track down the requested reconciliation time, as well as any other metric that can help us define the benchmark.

I'd just say something regarding the maximum amount of resources, as Tenants or Namespaces: this is strictly related to Kubernetes itself, and more specifically, to etcd. We're storing the CRDs definitions into it, so we don't have any hard limit since it's something out of our boundaries.

[bsctl]

@ptx96 is it possible to use your experience on #300 to provide a benchmark for Capsule performances?

[bsctl]

@ptx96 please take care of defining PrometheusRule for the Alert Manager. Aims is to inform the cluster admin about possible critical issues about Capsule, for example:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: capsule
spec:
  groups:
  - name: capsule-failures
    rules:
    - alert: CapsuleFailures
      annotations:
        message: Capsule reconcile takes too long ....
      expr:
      for: 15m
      labels:
        severity: warning