Colima runs should never overwrite configuration on disk

[purajit]

trafficstars

Description

Currently, running colima start with any configuration overwrites the profile on disk:

➜  colima start > /dev/null 2>&1 && yq .memory ~/.colima/default/colima.yaml
2

➜  colima stop  > /dev/null 2>&1 && colima start -m 4  > /dev/null 2>&1 && yq .memory ~/.colima/default/colima.yaml
4

➜  colima stop  > /dev/null 2>&1 && colima start -m 8  > /dev/null 2>&1 &&  yq .memory ~/.colima/default/colima.yaml
8

➜  colima start

# within container
$ free -h | head -2 | tail -1 | awk '{print $2}'
7.7Gi

This is highly undesirable unless explicitly asked for (for instance, maybe with a --save-config), since this means that a run of just colima start implies running the previous configuration, rather than running a profile you had saved. The point of a profile is to have a consistent saved config you can rely on, and any CLI args should be assumed to be overrides just for that specific run.

This is especially true when you're not using a profile, where the assumption of running colima start with no args would be to choose the documented defaults (for example, 2GB of memory).

(I also understand here that there might be a split in opinions on what the assumed behavior is, in which case I'd request at least having an option to prevent overwriting the configuration on disk, even if it doesn't default to that)

If you are interested in this issue, kindly react to the proposed solution https://github.com/abiosoft/colima/issues/613#issuecomment-2294682560.

[abiosoft]

I can understand your point.

Yeah, Colima basically merges the CLI flags (if specified) with the current config to generate a new config file, and thereafter starts off the (newly generated) config file.

It is kinda convenient and works for most people.

(I also understand here that there might be a split in opinions on what the assumed behavior is, in which case I'd request at least having an option to prevent overwriting the configuration on disk, even if it doesn't default to that)

I have no objections with that. What shall we call it? --temporary-config, --discard-config, --override-config=false, --save-config=false ? Naming things is hard :|

I would prefer to have a flag that the zero value defaults to the current behaviour i.e specifying it would disable overwriting the config file.

[purajit]

I would prefer to have a flag that the zero value defaults to the current behaviour i.e specifying it would disable overwriting the config file.

Makes sense, I'm not too opinionated on that

What shall we call it?

Again, no strong opinions. I think --override-config=false or --save-config=false are the most obvious.

I haven't dug into the codebase, but how reasonable would it be to have this flag in the profile config as well, indicating that that profile should never be overwritten by CLI?

A use case would be to distribute a colima profile for everyone working in an org with some canonical configs, which won't unintentionally be changed when someone's running some experiments. If it's something you're ok with but can't prioritize, I can give it a shot.

[abiosoft]

how reasonable would it be to have this flag in the profile config as well, indicating that that profile should never be overwritten by CLI?

Simply having a locked config might suffice.

# Prevent this config file from being updated by user specified CLI flags and temporary edits via the `--edit` flag.
# By default, Colima updates the config file to persist the changes specified via CLI flags or temporary edits.
# Default: false
locked: false

The other question woud be the behaviour of the --edit flag, should it be handled in similar fashion like the sample yaml snippet above?

[purajit]

Sorry, had a busy week.

I'm happy with that configuration, and I think it makes sense to prevent --edit from altering it as well.

Btw another behavior I've noticed, which may impact even people who don't want to lock their profiles - any additional comments written get erased. So for instance, even without changing any configs, if my original config had comment explanations for why a certain default was changed, it gets erased on a plain colima start.

[pangdaxing23]

The current behavior is extremely jarring and reads as a bug rather than a feature. Please do not touch the config unless it is asked for. And it is very natural for most people to assume overwriting the config would be opt-in rather than opt-out, so if you go with --save-config as the flag, I would vote that it should default to false . A simple colima start should simply run the default config.

[kustodian]

Are there any updates if this is going to be implemented? Because Colima always overwrites the configuration file it's hard to automate the Colima configuration.

[abiosoft]

It originated from the idea that the config file can double as documentation and always stay up to date with the latest configuration options as Colima evolves.

It also provides an easy way (from the implementation perspective) to merge the start flags and configuration. So basically, the flags are combined with the exisitng configuration to generate a new configuration file. That is how subsequent colima start works without requiring the user ro re-specify previously specified flags.

While that approach works for most users, there are a subset of users that are affected including the following

• users that leave extra comments in the config file
• users that rearrange the config structure for convenience
• users that already autogenerate the config file
• users that store the config file in a readonly filesystem

There is a workflow that Colima users are accustomed to and would prefer not to alter.

• Not having to re-specify the start flags
• Running colima start --edit to edit configuration on the fly and get it applied to the config file.

Proposal

My proposed solution would follow the suggestion and introduce a --save-config flag that would clearly indicate to the user that the flags would be combined with the current config, regenerated and persisted.

The --save-config flag would default to true for backward compatiblity and turning it off would require specifying --save-config=false.

What's next?

I would be happy to leave this open for some days to get some feedback before proceeding with the implementation.

[mirza-src]

The suggested proposal is exactly what I was thinking of but I would also like to add that currently the configuration file is overwritten even when the profile is already running, this behaviour doesn't seem intuitive to me and maybe it would be better to only update the configuration file if the profile is to be started.

[purajit]

I like the proposal, but I'd still say I'm curious how the users who want the config to be overwritten work - considering that this is the only CLI I've come across where one run of the command changes the behavior for all subsequent runs of the command, and where you aren't allowed to put comments in your config, or structure it how you want, be able to see which configs differ from defaults, etc.

All that and my comments below being said, I'll take the fastest/least friction route to getting this done, and I understand there is some behavior people are used to. I also understand that you're an individual building a tool that you use a specific way that has become popular, and I don't want to make it seem like I'm trying to force you to change your own tool and how you work - just curious about the quirk and seeing if you're comfortable with changing it.

The --save-config flag would default to true for backward compatiblity and turning it off would require specifying --save-config=false

I'd suggest using an environment variable, so that people don't have to remember to use --save-config each time. Otherwise, you could spend quite a bit of time building up a config, and then the first time your forget to specify the flag (which is reasonable to forget), you've lost all that work.

It would also allow people to keep a config in-repo and symlink it - otherwise, you're forced to copy the file since symlinking means the moment someone uses start, the config is changed; but without symlinking it's difficult to make sure collaborators are using the preferred, latest configuration.

can double as documentation and always stay up to date with the latest configuration options as Colima evolves

This is reasonable and actually a pretty great thing missing with other tools, but it could be done by just populating a template/example config that people can use. Even as it stands, this doesn't necessarily work perfectly since if you have multiple profiles, the configs go "out of sync" - only the profile that's used gets updated to the "latest", but not the others, so you can't rely on it for documentation until you've already started it.

It also provides an easy way (from the implementation perspective) to merge the start flags and configuration. So basically, the flags are combined with the exisitng configuration to generate a new configuration file. That is how subsequent colima start works without requiring the user to re-specify previously specified flags

This can be done on-the-fly: load the config file, overwrite the values as already done in prepareConfig, and just use the resulting Config instead of writing it and then reading the file again via configmanager.

[abiosoft]

I'd suggest using an environment variable, so that people don't have to remember to use --save-config each time. Otherwise, you could spend quite a bit of time building up a config, and then the first time your forget to specify the flag (which is reasonable to forget), you've lost all that work.

Yeah, you would be able to do that by setting COLIMA_SAVE_CONFIG=0.

[abiosoft]

This is part of the latest release https://github.com/abiosoft/colima/releases/tag/v0.7.4.

[purajit]

Amazing, thanks!

[kustodian]

What about disabling to save the config when starting colima with brew services start colima? Where can I set up the variable so that brew services picks it up (especially on reboot)?

[abiosoft]

@kustodian is there a way to pass environment variable to brew services?