feature request reload only on fields change inside annotated secret

[andrusstrockiy]

Is it possible to extend reloader so it will reload my pod in case no only on whole secret change but only on certain fields inside my secret ?

[rasheedamir]

what is a usecase @andrusstrockiy? why would you need that?

[andrusstrockiy]

Sorry for long reply. Here is more detailed case. Well i had an around 40 java microservices i.e pods and each of them had separate database schemas configured through jdbc connection string like that

jdbc:psql://psql-server.com:5432./database_of_ms_java_1 

which passed to container through env value of SPRING_DATASOURCE_URL from my foo-secret

Now instead of creating separate secret for each 40 pods we created one secret (foo-uber-secert) which holds all jdbc connections strings for readability and ease of maintenance. So my next step was to annotated all my 40 deployments with foo-uber-secret so reloader kept watching and restart pods in case of foo-uber-secret get changed.It works.

However with such automation we got another drawback for case when changes applied only to single value of jdbc connection string with app number 32 i.e so it was replaced from

jdbc:psql://psql-server.com:5432./database_of_ms_java_32

to something like

jdbc:psql://psql-new-server.com:5432./new-database_of_ms_java_32

In the end I got a cascade of restarts of all 40 pods but my changes were made only for application number 32 and only that deployment should be changed.

[rasheedamir]

@andrusstrockiy interesting; with uber-secret you will have this issue

Just wondering from design perspective how to cater this feature? @faizanahmad055 any thoughts?

[faizanahmad055]

It is achievable, but the usability might get a bit complicated. Although, the usability factor can be discussed. Since we can have a lot of secret keys inside a secret resource (same is true for configmap as well), and secret keys can have same names as well within different secret resources, it can get a bit complicated in identifying which secret key should trigger a restart in case of duplicate names. For example, if we have two secret resources by the name of db-credentials and mysql-credentials. And within both of these secrets, we can have secret like mysql-password: test. To avoid the conflict here, we need to use some kind of map. So we can either have to use an annotation that will contain a map of secret-resource name with the secret-key name. Or we have to introduce a configmap for reloader where we can put this mapping. As for the functionality of reloader is concerned, currently, we compare the whole value of secret/configmap before and after the change. To implement this enhancement, we have to compare each key-value pair in secret/configmaps' data before and after the change.

[andrusstrockiy]

@rasheedamir

with uber-secret you will have this issue

yeah that's why i'm bothering you

So we can either have to use an annotation that will contain a map of secret-resource name with the secret-key name.

@faizanahmad055

Well i can supply exact key and the name of my uber-secret in annotation on which reloader can reference

[rasheedamir]

As for the functionality of reloader is concerned, currently, we compare the whole value of secret/configmap before and after the change. To implement this enhancement, we have to compare each key-value pair in secret/configmaps' data before and after the change.

@faizanahmad055 is it viable to implement? any issues with performance?

[faizanahmad055]

It is doable. But it is a very specific usecase, where we have to take care of lot of corner cases. (i.e. the other deployments using the same secret does not rollout an update in case a secret key is added or deleted etc.). And we also have to make sure that it is backward compatible. If the current time cost is "n", the new time cost will be increased by a factor of "m" where "m" is the number of secret keys in a secret and "n" is the number of secrets (i.e. new time complexity = m x n).