proposal: allow all values to be compared for equality

[rogpeppe]

Currently, the specification says that only some kinds of value may be compared for equality.

For the record, the current rules are these:

In any comparison, the types of the two operands must unify or one of the operands must be null.

The equality operators == and != apply to operands that are comparable. [...]

• Null is comparable with itself and any other type. Two null values are always equal, null is unequal with anything else.
• Boolean values are comparable. Two boolean values are equal if they are either both true or both false.
• Integer values are comparable and ordered, in the usual way.
• Floating-point values are comparable and ordered, as per the definitions for binary coded decimals in the IEEE-754-2008 standard.
• Floating point numbers may be compared with integers.
• String and bytes values are comparable and ordered lexically byte-wise.
• Struct are not comparable.
• Lists are not comparable.

Note that the above description isn't entirely self-consistent: "floating point numbers may be compared with integers" conflicts with "the types of the two operands must unify or one of the operands must be null" because float does not unify with int.

It also isn't entirely complete: it does not mention that the arguments must be concrete, although the implementation requires that in most cases.

The current implementation does not correspond exactly to the above rules. In particular, it allows comparison of any value, including noncrete values, against bottom (_|_). The bottom-comparison semantics seem to be as follows:

• The literal bottom value (_|_) is comparable with any other type, and is equal if the other value is an error or is non-concrete.

There is at least one part of the standard library that implicitly assumes comparability between non-comparable values https://pkg.go.dev/cuelang.org/[email protected]/pkg/list#UniqueItems works on struct items, as it must because it's used to implement jsonschema semantics. However, the equality that it implements is shaky at best.

I propose that CUE defines equality between arbitrary values, using the JSONSchema instance-equality spec as a starting point.

To rephrase that spec in CUE terms:

Two CUE instances are said to be equal if and only if they have the same concrete value. Specifically, this means:

Before comparison, all defaults are resolved and optional fields omitted.

• both are null; or
• both are true; or
• both are false; or
• both are strings, and are the same byte-for-byte; or
• both are bytes, and are the same byte-for-byte; or
• both are numbers, and have the same mathematical value; or
• both are lists, and have an equal value item-for-item; or
• both are structs, and each field in one has exactly one field with a key equal to the other's, and that other field has an equal value.

Open questions:

• What should the result of comparing noncrete values be? Both false and and error seem like plausible answers here.
• What should the result of comparing against bottom be? The current semantics remain possible, but probably aren't desirable. We could disallow comparison against bottom entirely in favour of exists and isconcrete.

[mpvl]

Some additional clarifications:

• Comparing noncrete values should be a resolvable "incomplete" error. This is necessary to maintain compatibility with the lattice/ evaluator.
• Comparing two concrete values of a different type results in false, instead of an error. Maybe this is implied by the reference, but wanted to call that out explicitly. This would be a language change, but Unity reveals that this change cause no regressions.
• This proposal does not cover required fields: I suggest required fields should be treated as an incomplete error, similar to a noncrete value.
• As an extension of this proposal, we would also introduce == as a unary operator, which is now meaningful. This eliminates == as the only comparator for which there is no unary form.
• I agree that comparing against bottom should be abandoned. The meaning is quite unclear. But that should probably be change in a separate change.

An implementation of this proposal that can be used for experimentation is here: https://review.gerrithub.io/c/cue-lang/cue/+/1217011/1

This addresses several of the inconsistencies mentioned in this proposal in addition to implementing the changes.

[mvdan]

@rogpeppe @mpvl is this proposal fully implemented now, then? Should we close this now?

[mpvl]

Technically it is still an experiment. Depends on what convention we want to use. Up to you.

[mpvl]

Also, we haven't updated the spec yet. So there are some outstanding issues.

[mpvl]

Since there have been no objections, we will accept this proposal.