Thoughts on scaling kine

[tamalsaha]

Thanks for building this @ibuildthecloud !

• After reading up on the etcd data model, I wonder if it will be possible to run kine in a HA setup. Have you tried that? Any thoughts?

• Do you think it will be possible to store different data types in different tables so that it can scale well. Also, I am only curious about Postgres. Have you tried using jsonb data type so one can write reach queries against the table directly?

[liorokman]

Do you think it will be possible to store different data types in different tables so that it can scale well. Also, I am only curious about Postgres. Have you tried using jsonb data type so one can write reach queries against the table directly?

If you're only interested in PostgreSQL, you can store different data types in different tables without touching the Kine code. You just need to leverate PostgreSQL's partitioned tables. You can pre-create the kine table as a range partitioned table, and create separate partitions for each object type - as encoded in the name column. For example, a ranged partition from /registry/namespaces/ to /registry/namespaces0 will contain all the namespace objects.

Using jsonb would not work - Kubernetes actually uses a binary protobuf encoding for storing core objects. Only CRD instances are stored as JSON. Not to mention that when the API server is configured to encrypt a resource (e.g. a Secret), the object is encrypted, the data stored in the database is encrypted, and therefore would be unreadable, even if it were in JSON.