Default should be the default

[aspiwack]

Is your feature request related to a problem? Please describe. I've been thinking about overridability in the recent weeks. On and off, really. Nothing really concrete. Anyway, there's been a problem that's been bothering me for a while.

The way we plan on handling overridability is: we have a notion of default value, and if someone comes with a definite value, it's going to be used instead. That's neat, this is how Cue works as well, this squares well with the documentation as well, which will print: “default: false”.

But the problem is that we need to design fields for being overridden. By which I mean that any field which is not explicitly marked with default will consider its value final. That's bad! Because really, a field which is not overridable is considered a misfeature (this has been discussed in #218 , I believe). It shouldn't be a bad practice to use the default!

Describe the solution you'd like

Therefore, it is my current belief that, defining the value of a field should mean default, and definite/final values should be explicitly specified as such (in fact, there should probably be no definite value, and we probably want to specify a priority one way or another, but it's a somewhat orthogonal concern).

[yannham]

But the problem is that we need to design fields for being overridden. By which I mean that any field which is not explicitly marked with default will consider its value final. That's bad! Because really, a field which is not overridable is considered a misfeature (this has been discussed in #218 , I believe). It shouldn't be a bad practice to use the default!

Yes, I think we more or less all agree that overridable should be the default. Actually I've never been entirely convinced that just overriding default values would be sufficient, for the reason you cite, and just think of "normal/default" as a very simplistic binary priority system with two values. "normal" happens to be final, but just because it's top, and we could imagine add a "force" over it for example.

I feel like the question is: what to do when two non-mergeable values have the same priority, or how to ergonomically make this case avoidable in practice ? I see two approaches:

1. unbounded priorities: say, integers priorities, so that you can always set a priority higher or lower than an existing one.
2. overriding operation: add a // operator, as in Nix. Internally this would just be a right-biased merge, otherwise working in exactly the same way.

What I like about the second approach is that:

• (a) this clearly distinguishes non commutative overriding from merging. Merge is already quite overloaded (applying contracts, merging records recursively, setting default value), so keeping a "clean", commutative merge that can only override default values from schemas (keeping this simple two-value priority system), versus a monkey-patching one that can override any value of actual configurations, is, I think, not a bad idea. Even if in the end there's no real language-level distinction between an "actual configuration" and a "contract" (as record with metavalues), I imagine there is one in the user's head.
• (b) this is local: you don't have to know beforehand the chain of priorities in a sequence of merge, and you can't affect subsequent sequences of merge downstream by changing a priority. You just perform a functional record update, but on steroids once we do the right thing about merging recursive records.

An advantage of 1. is that you can set a higher priority in a record to force the propagation of a specific a value in every following merges happening downstream, which sounds a bit bad in general, but I guess could be useful for specific cases or for debugging. But even then, I would argue that we can still favor (b) and just add a third priority force over normal to achieve the same thing.

[thufschmitt]

Thinking with a NixOS-modules-like use-case, I'm not sure that 2. would be enough as the user doesn't control the order in which things will get merged (the actual merging process is a low-level implementation detail, everything the user should be concerned of is that the system will compute a big fixpoint over all the modules)

[yannham]

I see. So maybe we should support both, such that standard usage can still use a simple overriding operator, without depriving other users from thix NixOS modules approach.

[thufschmitt]

Note that (assuming you've the possibility to map over a record and manipulate meta-values a bit) you can roughly implement 2. in terms of 1. by doing something like a // b = merge a (map makeHighPrio b)

[yannham]

You're right, even if the semantics would be slightly different, because the // I propose would keep the priority unchanged when both sides have the same priority while your solution does not, although it probably doesn't matter in practice. However I think this is not only a question about what you can encode, but rather what do we want to make the idiomatic way. I don't see a problem in having two ways of doing something which overlap a bit, as long as you make clear that one is "the simple sufficient default stuff for most cases" and the other as "but if you need to do that, you can use this". Then you have to use them consistently in your examples/documentation/communication/whatever.

[yannham]

I think the original problem of this issue has been answered by RFC001 (#330), in particular with the new priorities and the recursive (or push) priorities operators.