cue icon indicating copy to clipboard operation
cue copied to clipboard

Published 20 hours ago • verified publisher icon cue-lang

evaluator: SIGSEGV / Non-consistent output on list comprehension constraint

Open exlee opened this issue 1 year ago • 5 comments

TL;DR; Found a scenario where depending on clause order cue evaluates correctly, gives unhelpful error or crashes.

Not sure if title describes well the context but examples should clear that out. Let me know if there's anything I could help with.

What version of CUE are you using (cue version)?

$ cue version
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS darwin
         GOARM64 v8.0

Does this issue reproduce with the latest stable release?

Yes

What did you do?

I tried to constraint dependency to known IDs while toying with GitHub Actions ID constraints. I got SIGSEV at the beginning but got same code working after changing clause order.

When dependency in spec wasn't optional cue evaluation didn't crash but unhelpful error was returned.

Scenario 1: unspecified deps, order spec -> iter, outcome: as expected

Input

$ testscript -v <<'EOD'
exec cue version
exec cue eval ./example.cue

-- example.cue --
#Item: {
  id!: string
  deps: [...or(known_ids)] | or(known_ids)
}

known_ids: [for k,v in items { v.id } ]

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]

EOD

Output

[ENV REDACTED]

> exec cue version
[stdout]
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS darwin
         GOARM64 v8.0
cue.lang.version v0.10.0
> exec cue eval ./example.cue
[stdout]
#Item: {
    id!: string
    deps: [] | "a-id" | "b-id" | "c-id"
}
known_ids: ["a-id", "b-id", "c-id"]
items: {
    a: {
        id:   "a-id"
        deps: "b-id"
    }
    b: {
        id: "b-id"
        deps: []
    }
    c: {
        id: "c-id"
        deps: ["a-id", "b-id"]
    }
}
PASS
Scenario 2: unspecified deps, order iter -> spec, outcome: error

Input

$ testscript -v <<'EOD'
exec cue version
exec cue eval ./example.cue

-- example.cue --
known_ids: [for k,v in items { v.id } ]

#Item: {
  id!: string
  deps: [...or(known_ids)] | or(known_ids)
}


items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]

EOD

Output

[ENV REDACTED]

> exec cue version
[stdout]
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS darwin
         GOARM64 v8.0
cue.lang.version v0.10.0
> exec cue eval ./example.cue
[stderr]
items.a: cannot add to field _:
    ./example.cue:1:32
[exit status 1]
FAIL: /tmp/testscript2638425491/scenario2.txt/script.txtar:2: unexpected command failure
Scenario 3: optional deps, order spec -> iter, outcome: as expected

Input

$ testscript -v <<'EOD'
exec cue version
exec cue eval ./example.cue

-- example.cue --
#Item: {
  id!: string
  deps?: [...or(known_ids)] | or(known_ids)
}

known_ids: [for k,v in items { v.id } ]

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]
items: d: _

EOD

Output

[ENV REDACTED]

> exec cue version
[stdout]
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS darwin
         GOARM64 v8.0
cue.lang.version v0.10.0
> exec cue eval ./example.cue
[stdout]
#Item: {
    id!: string
}
known_ids: ["a-id", "b-id", "c-id", "d-id"]
items: {
    a: {
        id:   "a-id"
        deps: "b-id"
    }
    b: {
        id: "b-id"
        deps: []
    }
    c: {
        id: "c-id"
        deps: ["a-id", "b-id"]
    }
    d: {
        id: "d-id"
    }
}
PASS
Scenario 4: optional deps, order iter -> spec, outcome: crash

Input

$ testscript -v <<'EOD'
exec cue version
exec cue eval ./example.cue

-- example.cue --
known_ids: [for k,v in items { v.id } ]

#Item: {
  id!: string
  deps?: [...or(known_ids)] | or(known_ids)
}

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]
items: d: _

EOD

Output

[ENV REDACTED]

> exec cue version
[stdout]
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS darwin
         GOARM64 v8.0
cue.lang.version v0.10.0
> exec cue eval ./example.cue
[stderr]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x1d0 pc=0x102b53f80]

goroutine 1 gp=0x140000021c0 m=4 mp=0x14000073008 [running]:
panic({0x10328e300?, 0x1038d9590?})
	runtime/panic.go:804 +0x154 fp=0x1400038dcd0 sp=0x1400038dc20 pc=0x102908144
runtime.panicmem(...)
	runtime/panic.go:262
runtime.sigpanic()
	runtime/signal_unix.go:900 +0x300 fp=0x1400038dd30 sp=0x1400038dcd0 pc=0x10290a3f0
cuelang.org/go/internal/core/adt.(*Vertex).updateArcType(0x140004fdd60, 0xe0?)
	cuelang.org/[email protected]/internal/core/adt/composite.go:328 +0x60 fp=0x1400038ddb0 sp=0x1400038dd40 pc=0x102b53f80
cuelang.org/go/internal/core/adt.(*Vertex).hasConjunct(0x140004fdd60, {0x140004048c0, {0x10337a918, 0x140003eea40}, {0x14000404780, 0x0, 0x0, 0x0, 0x0, 0x0, ...}})
	cuelang.org/[email protected]/internal/core/adt/composite.go:1196 +0xe4 fp=0x1400038de70 sp=0x1400038ddb0 pc=0x102b579a4
cuelang.org/go/internal/core/adt.(*nodeContext).notifyConjunct(0x140000df808, {0x140004048c0, {0x10337a918, 0x140003eea40}, {0x14000404780, 0x0, 0x0, 0x0, 0x0, 0x0, ...}})
	cuelang.org/[email protected]/internal/core/adt/composite.go:1266 +0xcc fp=0x1400038df00 sp=0x1400038de70 pc=0x102b5852c
cuelang.org/go/internal/core/adt.(*nodeContext).addConjunctDynamic(0x140000df808, {0x140004048c0, {0x10337a918, 0x140003eea40}, {0x14000404780, 0x0, 0x0, 0x0, 0x0, 0x0, ...}})
	cuelang.org/[email protected]/internal/core/adt/composite.go:1257 +0x188 fp=0x1400038df70 sp=0x1400038df00 pc=0x102b583c8
cuelang.org/go/internal/core/adt.(*nodeContext).insertField(0x1400009ec08, 0x4f1, 0x1?, {0x140004048c0, {0x10337a918, 0x140003eea40}, {0x14000404780, 0x0, 0x0, 0x0, ...}})
	cuelang.org/[email protected]/internal/core/adt/eval.go:2146 +0x2ac fp=0x1400038e080 sp=0x1400038df70 pc=0x102b8176c
cuelang.org/go/internal/core/adt.(*nodeContext).addStruct(0x1400009ec08, 0x140003ef4c0, 0x14000522a20, {0x14000404780, 0x0, 0x0, 0x0, 0x0, 0x0, {0x1, ...}})
	cuelang.org/[email protected]/internal/core/adt/eval.go:2073 +0x6dc fp=0x1400038e1d0 sp=0x1400038e080 pc=0x102b80d9c
cuelang.org/go/internal/core/adt.(*nodeContext).addExprConjunct(0x1400009ec08, {0x140003ef4c0, {0x10337a918, 0x140003eec00}, {0x14000404780, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1604 +0x408 fp=0x1400038e340 sp=0x1400038e1d0 pc=0x102b7d758
cuelang.org/go/internal/core/adt.(*nodeContext).addVertexConjuncts(0x1400009ec08, {0x140003efb80, {0x10337a878, 0x1400042a840}, {0x140003efba0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1816 +0x4dc fp=0x1400038e4b0 sp=0x1400038e340 pc=0x102b7e98c
cuelang.org/go/internal/core/adt.(*nodeContext).evalExpr(0x1400009ec08, {0x140003efb80, {0x10337a878, 0x1400042a840}, {0x140003efba0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1679 +0x65c fp=0x1400038e5e0 sp=0x1400038e4b0 pc=0x102b7e30c
cuelang.org/go/internal/core/adt.(*nodeContext).addExprConjunct(0x1400009ec08, {0x140003efb80, {0x10337a878, 0x1400042a840}, {0x140003efba0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1623 +0x464 fp=0x1400038e750 sp=0x1400038e5e0 pc=0x102b7d7b4
cuelang.org/go/internal/core/adt.(*nodeContext).insertConjuncts(0x1400009ec08, 0xc0?)
	cuelang.org/[email protected]/internal/core/adt/eval.go:413 +0xd4 fp=0x1400038e820 sp=0x1400038e750 pc=0x102b78394
cuelang.org/go/internal/core/adt.(*OpContext).unify(0x1400007c480, 0x140004fcf00, 0x2)
	cuelang.org/[email protected]/internal/core/adt/eval.go:247 +0xa1c fp=0x1400038ea80 sp=0x1400038e820 pc=0x102b7804c
cuelang.org/go/internal/core/adt.(*OpContext).lookup(0x1400007c480, 0x140004fd180, {0x1400026d570?, 0x0?}, 0x4d1, 0x410202)
	cuelang.org/[email protected]/internal/core/adt/context.go:946 +0x524 fp=0x1400038ebc0 sp=0x1400038ea80 pc=0x102b66a04
cuelang.org/go/internal/core/adt.(*FieldReference).resolve(0x14000496cb0, 0x1400007c480, 0x410202)
	cuelang.org/[email protected]/internal/core/adt/expr.go:718 +0x68 fp=0x1400038ec00 sp=0x1400038ebc0 pc=0x102b86538
cuelang.org/go/internal/core/adt.(*OpContext).unifyNode(0x1400007c480, {0x103382300, 0x14000496cb0}, 0x410202)
	cuelang.org/[email protected]/internal/core/adt/context.go:838 +0x1f4 fp=0x1400038ecb0 sp=0x1400038ec00 pc=0x102b65d24
cuelang.org/go/internal/core/adt.(*OpContext).node(0x1400007c480, {0x10337a9b8, 0x140003ee9c0}, {0x103382300, 0x14000496cb0}, 0x1, 0x0?)
	cuelang.org/[email protected]/internal/core/adt/context.go:1061 +0x44 fp=0x1400038ed60 sp=0x1400038ecb0 pc=0x102b67b34
cuelang.org/go/internal/core/adt.(*SelectorExpr).resolve(0x140003ee9c0, 0x1400007c480, 0x7fff0102)
	cuelang.org/[email protected]/internal/core/adt/expr.go:1007 +0x6c fp=0x1400038edb0 sp=0x1400038ed60 pc=0x102b874ac
cuelang.org/go/internal/core/adt.(*OpContext).resolveState(0x1400007c480, {0x140003efb00, {0x10337a9b8, 0x140003ee9c0}, {0x140003efb40, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/context.go:488 +0x104 fp=0x1400038eeb0 sp=0x1400038edb0 pc=0x102b63964
cuelang.org/go/internal/core/adt.(*nodeContext).evalExpr(0x1400009e608, {0x140003efb00, {0x10337a9b8, 0x140003ee9c0}, {0x140003efb40, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1647 +0x528 fp=0x1400038efe0 sp=0x1400038eeb0 pc=0x102b7e1d8
cuelang.org/go/internal/core/adt.(*nodeContext).addExprConjunct(0x1400009e608, {0x140003efb00, {0x10337a9b8, 0x140003ee9c0}, {0x140003efb40, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1623 +0x464 fp=0x1400038f150 sp=0x1400038efe0 pc=0x102b7d7b4
cuelang.org/go/internal/core/adt.(*nodeContext).addStruct(0x1400009e608, 0x140003ef6a0, 0x14000522990, {0x140003efb20, 0x0, 0x0, 0x0, 0x0, 0x0, {0x0, ...}})
	cuelang.org/[email protected]/internal/core/adt/eval.go:2097 +0xbc4 fp=0x1400038f2a0 sp=0x1400038f150 pc=0x102b81284
cuelang.org/go/internal/core/adt.(*nodeContext).addExprConjunct(0x1400009e608, {0x140003ef6a0, {0x10337a798, 0x14000522990}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
	cuelang.org/[email protected]/internal/core/adt/eval.go:1604 +0x408 fp=0x1400038f410 sp=0x1400038f2a0 pc=0x102b7d758
cuelang.org/go/internal/core/adt.(*nodeContext).insertConjuncts(0x1400009e608, 0xa0?)
	cuelang.org/[email protected]/internal/core/adt/eval.go:413 +0xd4 fp=0x1400038f4e0 sp=0x1400038f410 pc=0x102b78394
cuelang.org/go/internal/core/adt.(*OpContext).unify(0x1400007c480, 0x140004fd360, 0x7fff0105)
	cuelang.org/[email protected]/internal/core/adt/eval.go:247 +0xa1c fp=0x1400038f740 sp=0x1400038f4e0 pc=0x102b7804c
cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0x14000071808, 0x5)
	cuelang.org/[email protected]/internal/core/adt/eval.go:811 +0x188 fp=0x1400038f8c0 sp=0x1400038f740 pc=0x102b79988
cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0x14000071808, 0x5)
	cuelang.org/[email protected]/internal/core/adt/eval.go:595 +0x1d4 fp=0x1400038f990 sp=0x1400038f8c0 pc=0x102b78cf4
cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0x14000071808, 0x5, 0x14000071808, 0x0, 0x0, 0x1)
	cuelang.org/[email protected]/internal/core/adt/disjunct.go:178 +0x2d8 fp=0x14000390310 sp=0x1400038f990 pc=0x102b6f6e8
cuelang.org/go/internal/core/adt.(*OpContext).unify(0x1400007c480, 0x140004fcc80, 0x7fff0105)
	cuelang.org/[email protected]/internal/core/adt/eval.go:302 +0x50c fp=0x14000390570 sp=0x14000390310 pc=0x102b77b3c
cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0x14000071208, 0x5)
	cuelang.org/[email protected]/internal/core/adt/eval.go:811 +0x188 fp=0x140003906f0 sp=0x14000390570 pc=0x102b79988
cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0x14000071208, 0x5)
	cuelang.org/[email protected]/internal/core/adt/eval.go:595 +0x1d4 fp=0x140003907c0 sp=0x140003906f0 pc=0x102b78cf4
cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0x14000071208, 0x5, 0x14000071208, 0x0, 0x0, 0x1)
	cuelang.org/[email protected]/internal/core/adt/disjunct.go:178 +0x2d8 fp=0x14000391140 sp=0x140003907c0 pc=0x102b6f6e8
cuelang.org/go/internal/core/adt.(*OpContext).unify(0x1400007c480, 0x140004fcbe0, 0x7fff0405)
	cuelang.org/[email protected]/internal/core/adt/eval.go:302 +0x50c fp=0x140003913a0 sp=0x14000391140 pc=0x102b77b3c
cuelang.org/go/internal/core/adt.(*Vertex).Finalize(0x14000391418?, 0x1400007c480)
	cuelang.org/[email protected]/internal/core/adt/composite.go:822 +0x58 fp=0x140003913d0 sp=0x140003913a0 pc=0x102b55e48
cuelang.org/go/cue.newVertexRoot(0x1400027d4c0, 0x1039d45b8?, 0x140004fcbe0)
	cuelang.org/[email protected]/cue/types.go:602 +0x2c fp=0x14000391400 sp=0x140003913d0 pc=0x102d7481c
cuelang.org/go/cue.newValueRoot(0x0?, 0x14000391530?, {0x103382600?, 0x140004fcbe0?})
	cuelang.org/[email protected]/cue/types.go:611 +0x3c fp=0x140003914e0 sp=0x14000391400 pc=0x102d748ac
cuelang.org/go/cue.(*Context).make(0x1400027d4c0, 0x140004fcbe0)
	cuelang.org/[email protected]/cue/context.go:252 +0x84 fp=0x14000391550 sp=0x140003914e0 pc=0x102d657a4
cuelang.org/go/cue.(*Context).BuildInstances(0x1400027d4c0, {0x14000391820, 0x1, 0x1e0?})
	cuelang.org/[email protected]/cue/context.go:150 +0x1ac fp=0x14000391610 sp=0x14000391550 pc=0x102d64c6c
cuelang.org/go/cmd/cue/cmd.buildInstances(0x1400028c8a0, {0x14000391820, 0x1, 0x25a?}, 0x1)
	cuelang.org/[email protected]/cmd/cue/cmd/common.go:724 +0x38 fp=0x140003916c0 sp=0x14000391610 pc=0x1030273a8
cuelang.org/go/cmd/cue/cmd.parseArgs(0x1400028c8a0, {0x140002be120, 0x1, 0x1}, 0x1400027ed20)
	cuelang.org/[email protected]/cmd/cue/cmd/common.go:603 +0xa58 fp=0x14000391830 sp=0x140003916c0 pc=0x103026328
cuelang.org/go/cmd/cue/cmd.runEval(0x1400028c8a0, {0x140002be120, 0x1, 0x1})
	cuelang.org/[email protected]/cmd/cue/cmd/eval.go:88 +0x68 fp=0x14000391a20 sp=0x14000391830 pc=0x10302ba68
cuelang.org/go/cmd/cue/cmd.newEvalCmd.mkRunE.func1(0x140001c2900?, {0x140002be120, 0x1, 0x1})
	cuelang.org/[email protected]/cmd/cue/cmd/root.go:136 +0x20c fp=0x14000391c10 sp=0x14000391a20 pc=0x10302b56c
github.com/spf13/cobra.(*Command).execute(0x1400028ac08, {0x140002be0e0, 0x1, 0x1})
	github.com/spf13/[email protected]/command.go:985 +0x834 fp=0x14000391da0 sp=0x14000391c10 pc=0x102a823c4
github.com/spf13/cobra.(*Command).ExecuteC(0x1400028a308)
	github.com/spf13/[email protected]/command.go:1117 +0x344 fp=0x14000391e80 sp=0x14000391da0 pc=0x102a82b84
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:1041
cuelang.org/go/cmd/cue/cmd.(*Command).Run(0x1400028c8a0, {0x10393cea0?, 0x10328f2c0?})
	cuelang.org/[email protected]/cmd/cue/cmd/root.go:401 +0x28 fp=0x14000391ea0 sp=0x14000391e80 pc=0x10304d928
cuelang.org/go/cmd/cue/cmd.Main()
	cuelang.org/[email protected]/cmd/cue/cmd/root.go:309 +0x9c fp=0x14000391f20 sp=0x14000391ea0 pc=0x10304d49c
main.main()
	cuelang.org/[email protected]/cmd/cue/main.go:24 +0x1c fp=0x14000391f40 sp=0x14000391f20 pc=0x10306e91c
runtime.main()
	runtime/proc.go:272 +0x288 fp=0x14000391fd0 sp=0x14000391f40 pc=0x1028d28d8
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000391fd0 sp=0x14000391fd0 pc=0x102910cd4

goroutine 2 gp=0x14000002700 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006cf90 sp=0x1400006cf70 pc=0x1029084b8
runtime.goparkunlock(...)
	runtime/proc.go:430
runtime.forcegchelper()
	runtime/proc.go:337 +0xb8 fp=0x1400006cfd0 sp=0x1400006cf90 pc=0x1028d2c38
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006cfd0 sp=0x1400006cfd0 pc=0x102910cd4
created by runtime.init.7 in goroutine 1
	runtime/proc.go:325 +0x24

goroutine 3 gp=0x14000002c40 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006d760 sp=0x1400006d740 pc=0x1029084b8
runtime.goparkunlock(...)
	runtime/proc.go:430
runtime.bgsweep(0x1400007c000)
	runtime/mgcsweep.go:317 +0x108 fp=0x1400006d7b0 sp=0x1400006d760 pc=0x1028bcb28
runtime.gcenable.gowrap1()
	runtime/mgc.go:203 +0x28 fp=0x1400006d7d0 sp=0x1400006d7b0 pc=0x1028b0a58
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006d7d0 sp=0x1400006d7d0 pc=0x102910cd4
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:203 +0x6c

goroutine 4 gp=0x14000002e00 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x1031d28d0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006df60 sp=0x1400006df40 pc=0x1029084b8
runtime.goparkunlock(...)
	runtime/proc.go:430
runtime.(*scavengerState).park(0x103912640)
	runtime/mgcscavenge.go:425 +0x5c fp=0x1400006df90 sp=0x1400006df60 pc=0x1028ba4bc
runtime.bgscavenge(0x1400007c000)
	runtime/mgcscavenge.go:658 +0xac fp=0x1400006dfb0 sp=0x1400006df90 pc=0x1028baa6c
runtime.gcenable.gowrap2()
	runtime/mgc.go:204 +0x28 fp=0x1400006dfd0 sp=0x1400006dfb0 pc=0x1028b09f8
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006dfd0 sp=0x1400006dfd0 pc=0x102910cd4
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:204 +0xac

goroutine 17 gp=0x1400008a380 m=nil [finalizer wait]:
runtime.gopark(0x1400006c5b8?, 0x1028ac9f4?, 0x60?, 0xa0?, 0x33?)
	runtime/proc.go:424 +0xc8 fp=0x1400006c580 sp=0x1400006c560 pc=0x1029084b8
runtime.runfinq()
	runtime/mfinal.go:193 +0x108 fp=0x1400006c7d0 sp=0x1400006c580 pc=0x1028afb58
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006c7d0 sp=0x1400006c7d0 pc=0x102910cd4
created by runtime.createfing in goroutine 1
	runtime/mfinal.go:163 +0x80

goroutine 18 gp=0x1400008a700 m=nil [chan receive]:
runtime.gopark(0x14000068748?, 0x1029f11ec?, 0xd0?, 0xb1?, 0x103390340?)
	runtime/proc.go:424 +0xc8 fp=0x140000686f0 sp=0x140000686d0 pc=0x1029084b8
runtime.chanrecv(0x14000234310, 0x0, 0x1)
	runtime/chan.go:639 +0x414 fp=0x14000068770 sp=0x140000686f0 pc=0x1028a0274
runtime.chanrecv1(0x0?, 0x0?)
	runtime/chan.go:489 +0x14 fp=0x140000687a0 sp=0x14000068770 pc=0x10289fe24
runtime.unique_runtime_registerUniqueMapCleanup.func1(...)
	runtime/mgc.go:1732
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
	runtime/mgc.go:1735 +0x3c fp=0x140000687d0 sp=0x140000687a0 pc=0x1028b3b3c
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140000687d0 sp=0x140000687d0 pc=0x102910cd4
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
	runtime/mgc.go:1730 +0xa0

goroutine 19 gp=0x1400008ae00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x14000068f10 sp=0x14000068ef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x14000068fb0 sp=0x14000068f10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x14000068fd0 sp=0x14000068fb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000068fd0 sp=0x14000068fd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 20 gp=0x1400008afc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x14000069710 sp=0x140000696f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140000697b0 sp=0x14000069710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140000697d0 sp=0x140000697b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140000697d0 sp=0x140000697d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 21 gp=0x1400008b180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x14000069f10 sp=0x14000069ef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x14000069fb0 sp=0x14000069f10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x14000069fd0 sp=0x14000069fb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x14000069fd0 sp=0x14000069fd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 22 gp=0x1400008b340 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006a710 sp=0x1400006a6f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x1400006a7b0 sp=0x1400006a710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x1400006a7d0 sp=0x1400006a7b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006a7d0 sp=0x1400006a7d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 23 gp=0x1400008b500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006af10 sp=0x1400006aef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x1400006afb0 sp=0x1400006af10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x1400006afd0 sp=0x1400006afb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006afd0 sp=0x1400006afd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 24 gp=0x1400008b6c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006b710 sp=0x1400006b6f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x1400006b7b0 sp=0x1400006b710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x1400006b7d0 sp=0x1400006b7b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006b7d0 sp=0x1400006b7d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 25 gp=0x1400008b880 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x1400006bf10 sp=0x1400006bef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x1400006bfb0 sp=0x1400006bf10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x1400006bfd0 sp=0x1400006bfb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x1400006bfd0 sp=0x1400006bfd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 26 gp=0x1400008ba40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x140004a0710 sp=0x140004a06f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140004a07b0 sp=0x140004a0710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140004a07d0 sp=0x140004a07b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140004a07d0 sp=0x140004a07d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 27 gp=0x1400008bc00 m=nil [GC worker (idle)]:
runtime.gopark(0x31918e28b387a?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x140004a0f10 sp=0x140004a0ef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140004a0fb0 sp=0x140004a0f10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140004a0fd0 sp=0x140004a0fb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140004a0fd0 sp=0x140004a0fd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 28 gp=0x1400008bdc0 m=nil [GC worker (idle)]:
runtime.gopark(0x31918e28b8c22?, 0x3?, 0x7c?, 0x23?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x140004a1710 sp=0x140004a16f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140004a17b0 sp=0x140004a1710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140004a17d0 sp=0x140004a17b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140004a17d0 sp=0x140004a17d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 29 gp=0x140004a4000 m=nil [GC worker (idle)]:
runtime.gopark(0x10393e660?, 0x1?, 0x53?, 0x1?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x140004a1f10 sp=0x140004a1ef0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140004a1fb0 sp=0x140004a1f10 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140004a1fd0 sp=0x140004a1fb0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140004a1fd0 sp=0x140004a1fd0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140

goroutine 30 gp=0x140004a41c0 m=nil [GC worker (idle)]:
runtime.gopark(0x10393e660?, 0x1?, 0xe9?, 0x6d?, 0x0?)
	runtime/proc.go:424 +0xc8 fp=0x140004a2710 sp=0x140004a26f0 pc=0x1029084b8
runtime.gcBgMarkWorker(0x14000234540)
	runtime/mgc.go:1363 +0xdc fp=0x140004a27b0 sp=0x140004a2710 pc=0x1028b2dac
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1279 +0x28 fp=0x140004a27d0 sp=0x140004a27b0 pc=0x1028b2c98
runtime.goexit({})
	runtime/asm_arm64.s:1223 +0x4 fp=0x140004a27d0 sp=0x140004a27d0 pc=0x102910cd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1279 +0x140
[exit status 2]
FAIL: /tmp/testscript1325360298/scenario4.txt/script.txtar:2: unexpected command failure

What did you expect to see?

Scenario 1/3 outputs respectively.

What did you see instead?

As in output:

  • Scenario 2 includes returns error
  • Scenario 4 includes crashes with SIGSEGV

Test scripts:

scenario1.txt scenario2.txt scenario3.txt scenario4.txt

exlee avatar Aug 16 '24 18:08 exlee

  1. Investigating the crash with testscript scenario4.txt:

It occurs in ./internal/core/adt/composite.go:320:func (v *Vertex) updateArcType(t ArcType)

// updateArcType updates v.ArcType if t is more restrictive.
func (v *Vertex) updateArcType(t ArcType) {
	if t >= v.ArcType {
		return
	}
	if v.ArcType == ArcNotPresent {
		return
	}
	s := v.state
	if (s != nil || v.isFinal()) && s.ctx.isDevVersion() { ### CRASHES HERE
		c := s.ctx
		if s.scheduler.frozen.meets(arcTypeKnown) {
			parent := v.Parent
			parent.reportFieldCycleError(c, c.Source().Pos(), v.Label)
			return
		}
	}
...

due to unprotected reference to s.ctx.isDevVersion().

This change will fix it:

-    if (s != nil || v.isFinal()) && s.ctx.isDevVersion() {
+    if s != nil && (v.isFinal() || s.ctx.isDevVersion()) {

The unprotected reference was introduced in the commit

149b2489 Marcel van Lohuizen Sun Mar 3 13:19:34 2024 +0100 internal/core/adt: initialize nodeContext in two phases, see

./internal/core/adt/composite.go:290:func (v *Vertex) updateArcType(t ArcType):

-    if s := v.state; s != nil && s.ctx.isDevVersion() {
+    if s := v.state; (s != nil || v.isFinal()) && s.ctx.isDevVersion() {

rudifa avatar Aug 18 '24 10:08 rudifa

  1. Investigating the error with testscript scenario2.txt:

After the fix, scenario4.txt produces the same error as scenario2.txt:

items.a: cannot add to field _:
    ./example2.cue:1:32

The error is emitted in ./internal/core/adt/composite.go:1094:func (n *nodeContext) notifyConjunct(c Conjunct):

func (n *nodeContext) notifyConjunct(c Conjunct) {
	unreachableForDev(n.ctx)

	for _, rec := range n.notify {
		arc := rec.v
		if !arc.hasConjunct(c) {
			if arc.state == nil {
				// TODO: continuing here is likely to result in a faulty
				// (incomplete) configuration. But this may be okay. The
				// CUE_DEBUG=0 flag disables this assertion.
				n.ctx.Assertf(n.ctx.pos(), Debug, "unexpected nil state")
				n.ctx.addErrf(0, n.ctx.pos(), "cannot add to field %v", arc.Label)
				continue
			}
			arc.state.addConjunctDynamic(c)
		}
	}
}

The code block if arc.state == nil {...} was inserted into func notifyConjunct, in commit

125e40bed  Marcel van Lohuizen  Wed Feb 1 16:16:57 2023 +0100  internal/core/adt: add tests for 2235

Add assertion and debug bypass of assertion to expose the
panics in the test as an error.

Issue #2235

@mpvl's expertise will be needed here to find out why Scenario 2: unspecified deps, order iter -> spec produces an error, as reported above by @exlee.

UPDATE Also, the comment in func notifyConjunct is misleading because CUE_EXPERIMENT and CUE_DEBUG are comma-separated lists of key-value strings so that setting envvar CUE_DEBUG=0 will cause a parse error in envflag.Parse.

rudifa avatar Aug 18 '24 11:08 rudifa

@exlee thanks for going to the effort of creating reproducers here, especially as testscripts, and thanks @rudifa for digging in!

I've combined all scenarios into a single reproducer for ease of reference.

What did you do?

# scenario 1
exec cue eval ./scenario1.cue

# scenario 2
exec cue eval ./scenario2.cue

# scenario 3
exec cue eval ./scenario3.cue

# scenario 4
exec cue eval ./scenario4.cue

-- scenario1.cue --
#Item: {
  id!: string
  deps: [...or(known_ids)] | or(known_ids)
}

known_ids: [for k,v in items { v.id } ]

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]

-- scenario2.cue --
known_ids: [for k,v in items { v.id } ]

#Item: {
  id!: string
  deps: [...or(known_ids)] | or(known_ids)
}


items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]

-- scenario3.cue --
#Item: {
  id!: string
  deps?: [...or(known_ids)] | or(known_ids)
}

known_ids: [for k,v in items { v.id } ]

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]
items: d: _

-- scenario4.cue --
known_ids: [for k,v in items { v.id } ]

#Item: {
  id!: string
  deps?: [...or(known_ids)] | or(known_ids)
}

items: [string]: #Item

items: [Key=_]: { id: (Key) + "-id" }

items: a: deps: items.b.id
items: b: deps: []
items: c: deps: [items.a.id, items.b.id]
items: d: _

-- stdout.golden --
#Item: {
    id!: string
    deps: [] | "a-id" | "b-id" | "c-id"
}
known_ids: ["a-id", "b-id", "c-id"]
items: {
    a: {
        id:   "a-id"
        deps: "b-id"
    }
    b: {
        id: "b-id"
        deps: []
    }
    c: {
        id: "c-id"
        deps: ["a-id", "b-id"]
    }
}

What did you expect to see?

Passing test.

What did you see instead?

# scenario 1 (0.012s)
# scenario 2 (0.010s)
> exec cue eval ./scenario2.cue
[stderr]
items.a: cannot add to field _:
    ./scenario2.cue:1:32
[exit status 1]
FAIL: /tmp/testscript1287423478/repro.txtar/script.txtar:5: unexpected command failure

scenario4 also fails with a SIGSEV per @exlee.

Noting however that this test does pass with the new evaluator. I will add a regression test to lock in that behaviour.

myitcv avatar Aug 19 '24 14:08 myitcv

@myitcv Indeed, the single reproducer passes with CUE_EXPERIMENT=evalv3.

However, I still think that the test condition in func updateArcType at composite.go:320 should be fixed:

-	if (s != nil || v.isFinal()) && s.ctx.isDevVersion() {
+	if s != nil && s.ctx.isDevVersion() {
		c := s.ctx
		if s.scheduler.frozen.meets(arcTypeKnown) {
			parent := v.Parent
			parent.reportFieldCycleError(c, c.Source().Pos(), v.Label)
			return
		}
	}

This would prevent SIGSEV crash that will occur with any scenario that produces the condition

s != nil == false && v.isFinal() == true

(this is how scenario4 with the default evaluator causes the crash).

I believe that we can drop the v.isFinal() term, because

  • if s != nil we can use s in the test and in the code block, whatever value v.isFinal() has
  • if s == nil we must not use s, whatever value v.isFinal() has

rudifa avatar Aug 19 '24 22:08 rudifa

Reopening for @mpvl or @cuematthew to reply to @rudifa's suggestion above in case that's the correct fix for evalv2.

myitcv avatar Aug 20 '24 04:08 myitcv

Marcel found the same fix as @rudifa for a different evalv3 panic in https://review.gerrithub.io/c/cue-lang/cue/+/1202885 :) So this should be resolved in master shortly.

mvdan avatar Oct 22 '24 13:10 mvdan