cue
cue copied to clipboard
cue-lang
internal/core/eval: equal disjunction elements not eliminated
What version of CUE are you using (cue version)?
$ cue version v0.4.3-beta.1
Does this issue reproduce with the latest release?
Yes.
What did you do?
I want to use value.FillPath, it works fine with a single value in struct:
var a = `filltest: {
key1: *1 | int
}
var b = `{
key1: *1 | int
}`
// a.FillPath(cue.ParsePath("filltest"), b)
filltest: {
key1: *1 | int
}
Also works fine with different values in an array:
var a = `filltest: {
key1: [1] | [...int]
}
`
var b = `{
key1: [2] | [...int]
}`
// a.FillPath(cue.ParsePath("filltest"), b)
filltest: {
key1: [1] | [2] | [...int]
}
But act strange with the same values in one array:
var a = `filltest: {
key1: [1] | [...int]
}
`
var b = `{
key1: [1] | [...int]
}`
// a.FillPath(cue.ParsePath("filltest"), b)
filltest: {
key1: [1] | [1] | [1] | [...int]
}
What did you expect to see?
Expectation:
var a = `filltest: {
key1: [1] | [...int]
}
`
var b = `{
key1: [1] | [...int]
}`
// a.FillPath(cue.ParsePath("filltest"), b)
filltest: {
key1: [1] | [...int]
}
What did you see instead?
Tried to add some unit tests in FillPath but got the same error:
got: {
a: {
b: [1] | [1] | [1] | []
}
}
want: {
a: {
b: [1]
}
}
/cc @mpvl
@FogDong Thanks for the report.
Could you provide a full program that demonstrates the issue, please? (For example, it's not clear how you're printing the values shown above).
Looked briefly at this with @mvdan and it looks like a bug unrelated to FillPath.
exec cue eval x.cue
-- x.cue --
package x
a: filltest: {
key1: [1] | [...int]
}
b: {
key1: [1] | [...int]
}
res: a & {
filltest: b
}
gives:
a: {
filltest: {
key1: [1] | []
}
}
b: {
key1: [1] | []
}
res: {
filltest: {
key1: [1] | [1] | [1] | []
}
}
What's strange here is that the equal elements [1] and not considered as such when it comes to rendering the result of cue eval. Which I think ultimately means those disjunction elements have not been eliminated.
Marking as related to disjunctions for @mpvl
Here's a simpler version that also illustrates the problem:
exec cue eval x.cue
cmp stdout expect
-- x.cue --
y: [1] | [...int]
y: [1] | [...int]
-- expect --
y: [1] | []
This fails because the actual output is:
y: [1] | [1] | [1] | []
The output of cue def for this input is also not ideal:
y: ([1] | [...int]) & ([1] | [...int])
Thanks for the quick reply!
I found this bug when I tried to fill in my reference, maybe skipping filling the reference is a workaround to this problem for me 🤔. Still looking forward to the fix ASAP!
BTW, could you tell me where is the source code for this problem? Maybe I can try to fix this.
@FogDong just following up on this, please can you provide a repro comparing v0.2.2 (the version which I assume works for you) and tip, showing exactly what breaks for you?
In the meantime, I'm going to make a guess at what broke from your perspective:
# v0.2.2
go get cuelang.org/[email protected]
go mod tidy
go run .
cmp stdout stdout.golden
# tip
go get cuelang.org/[email protected]
go mod tidy
go run .
cmp stdout stdout.golden
-- go.mod --
module mod.com
go 1.18
require cuelang.org/go v0.2.2
-- main.go --
package main
import (
"fmt"
"cuelang.org/go/cue"
"cuelang.org/go/cue/format"
"cuelang.org/go/cue/load"
)
func main() {
bps := load.Instances([]string{"."}, nil)
v := cue.Build(bps)[0].Value().Lookup("z")
node := v.Syntax(cue.ResolveReferences(true))
b, err := format.Node(node)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", b)
}
-- x.cue --
package x
x: [1] | [...int]
y: [1] | [...int]
z: x & y
-- stdout.golden --
[...int]
I suspect your expectation is that this test should pass. However, it does not pass and gives:
# v0.2.2 (0.406s)
# tip (0.396s)
> go get cuelang.org/[email protected]
[stderr]
go: upgraded cuelang.org/go v0.2.2 => v0.4.4-0.20220801114602-5a08d2f7a9a7
go: upgraded github.com/emicklei/proto v1.6.15 => v1.10.0
go: upgraded golang.org/x/net v0.0.0-20200226121028-0de0cce0169b => v0.0.0-20220425223048-2871e0cb64e4
go: upgraded golang.org/x/text v0.3.2 => v0.3.7
go: upgraded golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 => v0.0.0-20200804184101-5ec99f83aff1
go: upgraded gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 => v3.0.1
> go mod tidy
> go run .
[stdout]
[1] | [1] | [1] | [...int]
> cmp stdout stdout.golden
--- stdout
+++ stdout.golden
@@ -1,1 +0,0 @@
-[1] | [1] | [1] | [...int]
@@ -0,0 +1,1 @@
+[...int]
FAIL: /tmp/testscript4139106237/repro.txtar/script.txt:11: stdout and stdout.golden differ
(Note there is a bug relating to the result [1] | [1] | [...int] above, but the duplicates in that disjunction can be ignored because they don't affect the semantic result).
The reason this test fails is that since v0.2.2 disjunction elimination behaviour has changed. The notes in https://github.com/cue-lang/cue/releases/tag/v0.3.0-beta.5 discuss this:
Note that v0.2 solved things differently: if in the result of a disjunction one term subsumes another, then the latter could be dropped. So in v0.2, the answer would be
x: {a: int}, even in the absence of defaults. The problem with this approach is that it could lead to confusing answers and that the default could not always be accurately determined. A goal of v0.3 was to have a simpler story for disambiguation and let simplifications be merely a matter of optimization.
Relating this comment to my test case. In v0.2.2:
[1] | [...int]
reduced to:
[...int]
But in tip it remains as:
[1] | [...int]
@FogDong - it would still be good to understand how/why this breaks you. Thanks
Actually this is an issue in both v0.2.2 and v0.4.3, I created this issue in April, it's not an issue preventing us from updating the cue version.
I create a repro here:
go mod tidy -compat=1.17
go run main.go
cmp stdout stdout.golden
-- go.mod --
module test-cue/and
go 1.17
require cuelang.org/go v0.4.4-0.20220729051708-0a46a1624353
require (
github.com/cockroachdb/apd/v2 v2.0.1 // indirect
github.com/emicklei/proto v1.10.0 // indirect
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect
github.com/google/uuid v1.2.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/mpvl/unique v0.0.0-20150818121801-cbe035fff7de // indirect
github.com/pkg/errors v0.8.1 // indirect
github.com/protocolbuffers/txtpbfmt v0.0.0-20220428173112-74888fd59c2b // indirect
golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4 // indirect
golang.org/x/text v0.3.7 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
-- main.go --
package main
import (
"fmt"
"cuelang.org/go/cue"
"cuelang.org/go/cue/cuecontext"
"cuelang.org/go/cue/format"
)
var a = `filltest: {
key1?: [1] | [...int]
}
`
var b = `{
key1?: [1] | [...int]
}`
func main() {
ctx := cuecontext.New()
a := ctx.CompileString(a)
b := ctx.CompileString(b)
result := a
for i := 0; i < 3; i++ {
result = result.FillPath(cue.ParsePath("filltest"), b)
b = result.LookupPath(cue.ParsePath("filltest"))
}
s, err := format.Node(result.Syntax(cue.All(), cue.ResolveReferences(true), cue.DisallowCycles(true), cue.Docs(true), cue.Attributes(true)))
if err != nil {
panic(err)
}
fmt.Println(string(s))
}
-- repro.txt --
-- stdout.golden --
{
filltest: {
key1?: [1] | [...int]
}
}
And the result is like:
> go run main.go
[stdout]
{
filltest: {
key1?: ([1] | [...int]) & ([1] | [...int]) & ([1] | [...int]) & ([1] | [...int])
}
}
> cmp stdout stdout.golden
--- stdout
+++ stdout.golden
@@ -1,5 +1,5 @@
{
- filltest: {
- key1?: ([1] | [...int]) & ([1] | [...int]) & ([1] | [...int]) & ([1] | [...int])
- }
+ filltest: {
+ key1?: [1] | [...int]
+ }
}
FAIL: /var/folders/2n/0vszm3355n114dk2r_ln8vrh0000gp/T/testscript40035967/repro.txt/script.txt:3: stdout and stdout.golden differ
error running repro.txt in /var/folders/2n/0vszm3355n114dk2r_ln8vrh0000gp/T/testscript40035967/repro.txt
In my example code, I use a for loop to FillPath another value. In KubeVela, we'll have a user cue value as the base value, and KubeVela creates a new value, and try to fill the user base value. In every controller reconciliation, we'll try to fill the path for the base value if the step is still running. So if the base value is in the format [1] | [...int], the result will be complicated. And if the base value is complicated enough and as reconciliation grows, the cue value will explose.
Thanks for the repro. So the bug you are seeing is related to something I mentioned in passing earlier:
(Note there is a bug relating to the result
[1] | [1] | [...int]above, but the duplicates in that disjunction can be ignored because they don't affect the semantic result).
However what I said there isn't quite correct. Because this bug does prevent manifesting concrete values:
exec cue export x.cue
cmp stdout stdout.golden
-- x.cue --
x?: [1] | [...int]
x: [1]
-- stdout.golden --
{
"x": [
1
]
}
should pass but gives:
> exec cue export x.cue
[stderr]
x: incomplete value [1] | [1]
[exit status 1]
FAIL: /tmp/testscript949714628/repro.txtar/script.txtar:1: unexpected command failure
Leaving this marked as "disjunctions" so we pick it up as part of that work.
This one's priority is high since there's no workaround.
This is a user-facing issue because we have a command called vela debug, in which all the cue values will be rendered and displayed to our users. This issue will make the debug output full of & with the same struct (if the original array is rather big, which is quite common in a k8s scenario, the output is a nightmare..)
Hope to be fixed in v0.6.0.