Option to commit Otomi (and maybe Helm) defaults to Gitea

[staticvoid255]

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem

I sometimes get frustrated when trying to find what values Otomi is applying by default

Describe the solution you'd like A clear and concise description of what you want to happen.

I would like to have the option to render out and commit Otomi and Helm defaults to Gitea, so I can have a complete overview of what configuration is being applied without going through otomi-core code and Helm charts.

I would like there to be 3 modes:

• basic - only values modified by the user are shown in Gitea (see attached screenshot)
• advanced - all Otomi-managed values are shown in Gitea
• pro - fully rendered verbose values file including Otomi defaults and Helm chart defaults

Aside from general config visibility, this would have the additional benefit of allowing users visibility over changes to Otomi and/or Helm defaults that happen in the background, which will empower SRE's.

Additional context Add any other context or screenshots about the feature request here.

[Morriz]

It is not needed to "commit" values in order to have an interest in them. We have put a lot of effort into separating defaults from user input, for many reasons. So asking to merge the two in the values repo is not beneficial.

If I read you correctly, you want to know what the defaults are. So would a command like otomi values --defaults-only do the trick?

Btw, did you know that you can get only the user input with otomi values --files-only?

[Morriz]

Another flag is --exclude-secrets which makes it easy to port values from one cluster to another, without copying secrets.

[staticvoid255]

Can you please elaborate on some of these reasons? I think it would be great to give users the option to write verbose values manifests to Gitea, especially when it comes to maintaining an open record of changes to Otomi and Helm defaults (because it's all in source control).

To perform the same kind of analysis it seems you are proposing:

1. Upgrade Otomi
2. Pull values repo
3. Rewind to last commit before the upgrade
4. Render values file using CLI
5. Go to the latest commit
6. Render another values file using CLI
7. Do a diff

This of course requires you be on a machine with Otomi CLI installed, and that every user interested in tracking changes be experienced with the CLI - which is simply not realistic and not how things work with our current clients.

I would like to:

1. Upgrade Otomi
2. Look at the clear record of changes as a commit in Gitea

Hiding values (and changes to them) away from users in gotmpls and defaults.yaml and the values schema and only exposing them through local CLI/diffing operations (other than maybe combing through verbose Drone pipeline logs) is not friendly for SRE's. I also want to be easily able to share the reports of changes without each person having to perform their own diff using the steps I mentioned earlier.

[Morriz]

My reasoning used to be this:

If we would write all values, including defaults and derived, that could have side effects on a future update behaving differently. When a default changes between versions, and the user has not created a value for it, the upgrade will introduce the new behaviour. If we had written a value for it as you propose, then the new default would not come into effect.

So to me this comes down to expectation management. I believe a user should not find settings changed that they did not manipulate themselves.

Now, wether that paradigm has been correctly codified I am not certain, but this is my reasoning.

[Morriz]

About SREs not knowing how to operate otomi, that is another topic. Expecting to find each and every setting to be found in the values repo is one thing, but it might surely pay off to learn to find the defaults which are in plain sight in the values-schema.yaml of the versioned image.

Bottom line is that defaults differ across versions, and so the leading source is the schema.

[Morriz]

Btw, I am not proposing what you say I am proposing. I clearly offered an extra option next to the options already available: otomi values --defaults-only.

[staticvoid255]

Hmm, I get where you're coming from, but in many cases I'd like to modify or extend Otomi defaults, rather than completely overwrite them. Let's take Nginx for example - there are config snippets as part of the Otomi defaults for which it would be very valuable for me to be able to share with the team, and asking them to go through Otomi code is not a viable option in terms of workflow (especially when it comes to API and Console being closed-source). What I propose as an intermediate solution is to print a warning in Core that certain values have changed, replaced or added, and in the mean time give the user the option to expose as much of the config as they'd like. This would without doubt unlock a lot of power for SRE's using Otomi.

[Morriz]

But we already have that in core. Just run otomi values. Can we please not keep repeating ourselves?

[Morriz]

Also, changes from otomi migrate (such as you describe) are printed on the screen.

[staticvoid255]

closed as has been implemented