Packages instances documentation clarity

[btrepp]

https://cuelang.org/docs/concept/modules-packages-instances/#instances

This documentation implies

./root.cue

package ?
name: string

would unify with ./folder/b.cue

package ?
struct: { test: name}

However I couldn't seem to get the affect described, with any combination of running cue in the leaf folder, and mismatching or matching package names.

I'll accept this could be a me problem, but it could also be a docs bug, because instances are complex. I can't find anything in the discord, or the discussions where someone seems to be able to explain them, not that an example of their usage to help me understand.

At a minimum, this documentation that describes instances could have an explanation and an example. I'd be happy to contribute it, but I have no idea what it's actually trying to tell me currently

Note: I do have a cue.mod folder aswell

What version of CUE are you using (cue version)?

$ cue version
cue version v0.10.0

go version go1.23.0
      -buildmode exe
       -compiler gc
       -trimpath true
     CGO_ENABLED 0
          GOARCH arm64
            GOOS linux
         GOARM64 v8.0
cue.lang.version v0.10.0

Does this issue reproduce with the latest stable release?

Yes

What did you do?

Followed the description from the docs

What did you expect to see?

Files would unify

What did you see instead?

Error that name is undefined.

Extra notes

Perhaps instances only work for files in cue.mod/pkg? eg being exported as an external package, but then why doesn't it work in a folder that way?

I have tried to get cue for working with packages for a month or so trying to understand this.

At the moment, I am leaning to just use a 'flat folder for cue', and organise by My_folder_file.cue in that folder, as it's a just too complex otherwise. I am sure it's understood by those familiar with the language, but the mental model is just really hard to get out.

I've noticed this is a pretty common struggle among those trying to use cue, so hence raising as a bug, either in the module definition or at least in the docs. As I desperately want to use it, but its behaviours with folders is beyond my understanding.

I've also noticed two patterns for those working around this

1. Preprocess into a flat directory.
2. Use a shell command to dump all the files you need into the cli/import.

That would work, but ideally I would love to use cue modules properly, so I can contribute some modules others could use.

[jpluscplusm]

👋 Hi @btrepp! :-)

I agree - modules, package and instances do need some more docs aimed at "getting started" / "getting up to speed with the fundamental mechanisms that CUE makes available". I'm just in the middle of updating that specific guide to bring it into line with features exposed by recent versions of CUE, but that update didn't extend as far as making it newcomer focused. Writing /that/ guide is definitely high on my list, so please allow me to ask a few questions that will inform it, later!

So here's a txtar reproducer of a very simple instance in action:

exec cue mod init
cd foo
exec cue eval
cmp stdout ../out
-- root.cue --
package foo

name: string
-- foo/folder.cue --
package foo

struct: {
	test: name
}
-- out --
name: string
struct: {
    test: string
}

Running this with https://testscri.pt/ results in a pass, so we know that out matches the cue eval output:

$ testscript demo.txtar 
PASS

Can you extrapolate from that guide's example, and get a similar instance running locally?

If "no", please use the txtar link I gave you above to construct an example that shows us, explicitly and precisely, the problems you're having. Here's a daft example of doing that:

---

Here's my reproducer:

```
exec date
cmp stdout out
-- out --
Thu Jan 01 18:04:10 BST 1999
```

I expected it to pass, but instead I saw this:

```
$ testscript example.txtar
> exec date
[stdout]
Thu Oct 17 18:05:51 BST 2024
> cmp stdout out
diff stdout out
--- stdout
+++ out
@@ -1,1 +1,1 @@
-Thu Oct 17 18:05:51 BST 2024
+Thu Jan 01 18:04:10 BST 1999

FAIL: /tmp/testscript2519812758/example.txtar/script.txtar:2: stdout and out differ
```

Clearly, today *is* Jan 1, 1999 - so what's going wrong?

---

If "yes", then (a) excellent :-) ... and (b) I have a favour to ask!

I'd find it really really useful if you could mentally step back in the debugging process that led to you opening this issue, and try to figure out the gnarly sticking points that prevented you getting to a place of "aaaaah, that all works now!". Why, I guess, the words in the guide you found didn't lead you to a working example. And let me be really clear: this is 100% not because I want you to say "oh yes, I should have got THAT bit right!", but because understanding the newcomer-to-CUE (or newcomer-to-feature-X) experience, and where it falls down and could be improved, is such an incredibly valuable resource for those of us writing the CUE docs! If you were able to articulate the problems you had, that would be super, super useful! :-D

[btrepp]

Hi :). Thanks for your response and apologies taking me a while to get back to doing this, I went to other tools, but the itch got me an I am playing around with cue again. I am having a few bit more success playing with the language, and am still tempted.

So I can see on your example, that you change directory into foo, I suspect this would have worked for me, and I understand now, but at the time that's the load bearing line, that only makes sense once you know.

At the moment, my brain is landing on 'package' will help build you some sort of output file, and you do seem to be able to have multiple 'packages' in a directory, so that's good, say in k8s potentially I could have my 'service' package and my 'deployment', package (though this still might not be the best way of doing it).

One of the things that's stumping me at the moment, is that as soon as two packages appear, it's like the 'instancing' more or less falls over, and says, "well, you must provide me all the files now", rather than 'walk itself up the tree'.

Even specifying a package explicitly, isn't allowed.

package a
hello: world

package b
ping:pong

means cue eval in a directory doesn't work, but even running cue eval -p a complains there is two packages. I feel like I know that, that's why I said do that package a,

found packages "a" (a.cue) and "b" (b.cue) in "." To have multiple packages it seems you must track, the files involved in that package, outside of cue, to then tell cue about it. This is... odd?, and this is before even looking around how to 'cross load' a package.

Moving on-to the mental model as you asked

I think I discovered and want to use cue, from horrible yaml templating, and just the confusing soup most of that ends up in, I was almost writing DSL types in a language, to then export to yaml, but saw cue and thought awesome, an existing tool.

You will usually start with a simple file, and it appears to work well, but when solving complex problems I start to like to break each component down to a specific domain, so that a problem gets 'composed up', eg GitHub, well there's a small component there in git, and drilling down it might have commands, and then drilling right down there's a function that might load the HEAD file and get the branch name. In most projects you might see that looking like

project\main.c
project\module\xyz.c
project\module\sub\submodule\head.c

I think cues aiming to be different, and its model is slowly beginning to make sense, but it's kinda taking 'composition' and flipping it over. You might approach it as main.c is the file I want, but in cue the folder structures different. It's the different way of thinking that initially caused me grief (and I am still getting there).

aside string interpolation makes this confusing too, as you need to concrete everything before it interpolates, a more 'specific' lower information can't seem to provide that information, and the errors on it are weird. This was trying to invert my thoughts and go 'well at a top level', I don't know the concrete name, but if parts are provided (like a function) I could interpolate it, so I could provide the 'structure' here. Turns out that won't work, so I must provide the 'structure' via validation in a regex, and repeat it at the specific layer, but at least I get decent errors :).

So in the realm of trying to thinking like, higher folders are generic, lower folders are more 'instances', it starts to feel very 'root-folder' heavy, most things need to be defined at the top of a folder structure.

From there, I started thinking, well, maybe 'modules' and 'packages' make sense. I could abstract say Kubernetes 'definitions' into its own package, and then consume it in 'myapp' package. So common boilerplates in a share-able (even if via copy and pasting) situation. But this seems really difficult, I still don't have it working.

It would be great if there was examples that had

1. How to work with 2x packages in a module, that span over a few items. Eg on k8s, a a deployment package, a service package, and maybe even a 'common-types' package.
2. How to write modules so that they can be consumed in other modules, say I make my library 'btrepps k8s definitions', and the user is happy to 'curl https://github....' that into a folder, how do they consume that.

I am going to play a bit, and might do a write-up once I do figure out how to effectively work with packages, and also how to potentially output multiple files in an efficient way.

[myitcv]

I'll leave @jpluscplusm to triage as appropriate here, with https://github.com/cue-lang/docs-and-content issues as required.

[jpluscplusm]

Hi @btrepp 👋 Some quick points to help your CUE journey (captured not just for yourself, but also for some different pieces of documentation I'm planning!)

So I can see on your example, that you change directory into foo, I suspect this would have worked for me, and I understand now, but at the time that's the load bearing line, that only makes sense once you know.

The cd is totally optional, as many cue sub-commands accept package-related arguments (as per cue help inputs). Here's the same repro that I posted above, avoiding the cd:

exec cue mod init
exec cue eval ./foo
cmp stdout out
-- root.cue --
package foo

name: string
-- foo/folder.cue --
package foo

struct: {
	test: name
}
-- out --
name: string
struct: {
    test: string
}

cue eval in a directory doesn't work, but even running cue eval -p a complains there is two packages. To have multiple packages it seems you must track, the files involved in that package, outside of cue, to then tell cue about it.

I know what you mean by this ... and I'm happy to be able to tell you that you're wrong! 😄 The -p flag doesn't have anything to do with the CUE package that's being evaluated (its job is solely to do with which CUE package non-CUE should be placed inside), and what you needed to run was something like this repro:

exec cue eval .:A
cmp stdout out
-- a.cue --
package A

package_A: true
-- b.cue --
package B

package_B: true
-- out --
package_A: true

This is mentioned on https://cuelang.org/docs/concept/modules-packages-instances/#files-belonging-to-a-package, which is the guide I mentioned previously as being in need of a rewrite or partner doc "for beginners", given how fundamental the "package" concept can be when using CUE!

It would be great if there was examples that had

• How to work with 2x packages in a module, that span over a few items. Eg on k8s, a a deployment package, a service package, and maybe even a 'common-types' package.

Thanks for this suggestion - I'll see what we can do!

How to write modules so that they can be consumed in other modules, say I make my library 'btrepps k8s definitions', and the user is happy to 'curl https://github/....' that into a folder, how do they consume that.

The primary method of CUE module distribution is via a registry. The Central Registry is available: have a go with this tutorial and see how far you get :-) https://cuelang.org/docs/tutorial/publishing-modules-to-the-central-registry/.

[mvdan]

With @jpluscplusm's response above, and given it has been a few months now, I'll close this as resolved. Thanks!