Create WASM system for embedding Thema operations in a browser

[sdboyer]

The thema command offers (or will offer) basic Thema operations via CLI and HTTP. It'd be absolutely amazing if we could also represent those operations as something that was easily embedded in a webpage.

This'd clearly be WASM. I've never worked with WASM and don't know where to start, but if the CUE playground is feasible, this must be, too.

I'm picturing three linked text input boxes:

1. Lineage input, a text box that accepts raw CUE input, and a lineage can be pasted
   • Could also support a mode where e.g. the exemplars can be selected from a dropdown menu
   • Or just an arbitrary URL, which it'll try to fetch and populate the lineage input
2. Data input, a text box that accepts raw YAML or JSON or CUE
   • Or, again, an arbitrary URL to slurp data from
3. Operation/output, which includes controls over which operation to run (validate, validate-any, translate), and shows the result of performing that operation using the given lineage, with the given data.
   • Or, given that the set of operations is finite, closed, and small, it might be better to just always show the output of all operations. They can all meaningfully share the equivalent of a -v in the same way they do across the thema data subcommands. I suspect that being able to see how all three operations harmoniously relate will reinforce something about how Thema itself works in a way that could be missed when individually running thema data commands.

[sdboyer]

@myitcv i know you put together the CUE playground with WASM - any tips on where to start with this?

[verdverm]

https://github.com/mattn/golang-wasm-example - I think the wasm_exec.js is where the connection (complicated?) part happens. I haven't done Go -> WASM in a hot minute.

This may be of interest as well: https://github.com/hack-pad/hackpadfs

[sdboyer]

Excellent, will check those out, ty! i'm also excited by the possibility that hackpadfs could mean we could stop using laher/mergefs in various places

[verdverm]

I haven't tried it, but someone in my slack was speaking highly of it

[gedw99]

i used hackpadfs :) Yep it's really useful and fast.
I wanted my golang server packages that need a File system to automatically work when running on Web, Mobile and Desktop also. I use it within GIO: https://github.com/gioui for building cross platform gui. I am also starting to use Cue inside GIO. I am however pulling the Cue from the server. I intend to allow users to write their own Cue at runtime in order to augment the system around authz, data validation, workflows, migrations. Still very early days though.

I also use Cue Server side. I just run the Cue that the Users write in the front end at runtime.

---

@sdboyer BTW while i am here may i do a drive by request.
Perhaps add https://www.inkandswitch.com/cambria/ to the README under ##Prior Art. I understood thema much better by reading their uses cases and battles along the way towards best practice operating patterns etc.

[sdboyer]

@gedw99 thanks for the info! And yeah, i've been feeling a little Leibnizian about Cambria - i started work on all this before i knew it existed, then learned about it later. That, and Thema goes aggressively in a different direction with its insistence on linear ordering, were enough to make me back off a bit.

But it's just me being precious. I'll add a section to the README with Cambria, and some others.

[sdboyer]

voila https://github.com/grafana/thema#priorrelated-art

[gedw99]

@sdboyer cheers. thanks for the explanation. It makes total sense now that you told us. I was reading Thema and Cambria, and they looking like "same same but different" but i couldn't put my finger on the difference.

One of the associated domain problems with Schema Evolution that sort of goes hand in hand is data reconciliation in terms of CRDT. Have you ever considered Cue for doing CRDT ? Cue is definitely NOT a one trick pony !! I am playing with the idea. Both CRDT and CUE are based on semi lattice maths, non order dependent mutations and reconciliation. I have a sync system and using lamport vector clocks but i wonder if Cue is up to the challenge of CRDT style data sync without relying on virtual clocks / date time stamps for context. I know this is totally left field from Thema, but I figured if anyone is making jokes about Leibnizian, they will have an opinion on CRDT and CUE.

If you wanna chat offline about Data sync and thema it would be cool too. I really liked your video presentation where you framed the domain problem really well. Schema Evolution and front or back versioning approaches is something i have also been banging my head against and asking " is there is a better way ".

[sdboyer]

I figured if anyone is making jokes about Leibnizian, they will have an opinion on CRDT and CUE.

❤️

i've thought a bit about a potential relationship between CUE and CRDTs (and to some other lattice-related work i've done), though i've had trouble seeing one. I lack the formal maths knowledge to know how to express the breakdown well, but it starts somewhere around my intuitive sense that a prerequisite for creating a useful CRDT is the construction of a semilattice where all values in the lattice are meaningful. Because CUE's lattice is predetermined, and join operations all ultimately converge towards bottom - generally an unuseful value - the applicability isn't straightforward.

...but i don't find that convincing, either. Bottom occurs when you're directly banging two CUE values together: it indicates that the two values have reached a terminal state. That's only "unuseful" in a scenario where there is no larger, meaningful structure from which they derive, implying some recourse. And that really is the meaning of bottom - it tells the user where irreconcilable disagreement lives. And if you make the "user" a machine which knows that, when a given pair of values reach bottom, to take some new action that relates to where those values came from...i mean, then, you may have the basis for a CRDT. It seems plausible that you could model tombstone records e.g. in the Treedoc case using bottom.

So, yeah, possible. But i can't see far enough down that path to picture what i'd try to model. I'd like to, though - i think there's probably something valuable there for CUE's config-oriented domain.

If you wanna chat offline about Data sync and thema it would be cool too.

Sure! send me a DM on the CUE slack or le tweets