fermyon
Should `spin build` perform component dependency composition?
Currently when running spin build we do not perform component dependency composition. This means that developers won't know about issues with dependencies until they actually go to run the app. Ideally, we would give that feedback earlier.
On the flip side, the cost of component dependency composition is not trivial, and doing that composition to only just through that result away and do it again at load time is a bit wasteful. Would this lead us to think about storing the result of composition in some cache so that we don't necessarily have to do the composition at load time?
spin build doesn't validate a lot of stuff - for example, it doesn't check that the actual main components match the triggers they're assigned to! It doesn't check that Wasm files exist, or that registry references exist or are even well-formed. That stuff is all left to load time validation (or, if you're diligent, to spin doctor).
The original conception was that spin build was merely a minimal helper to save you running lengthy and intricate cargo or tinygo incantations by hand, and once your project grew up, you'd move on to a grown-up build system. So far, however, that hasn't turned out to be the case.
With those few years' experience, then, it's certainly reasonable to look at reinventing spin build to do a bunch more preparation and validation. Like this is the one way to build Spin applications, and if your app passes the one and only build gauntlet then it will run. We can add trigger/entry point validation, pre-fetch HTTP and registry components, etc. We could even emit the lockfile, maybe pre-composed by default like we do for OCI, at build time. Maybe build should even be implicit by default, like spin up / registry push / etc automatically run it, so that building, loading, and validating were unified - this would solve a bunch of challenges for Till's profiles SIP!
That's of course waaaay broader than this issue, but it kinda feels like the natural end state. After all, it would be odd if I fixed a composition error in build, and still got a runtime error that "uh your component doesn't export wasi:http/incoming-handler".
On the other hand, part of me still frets that we should separate the generic toolchain stuff from the validation stuff. If someone does choose to use make or msbuild (what? it could happen) to manage the generation of Wasm binaries, their makefile should still be able to call into Spin for the Spin-specific validation concerns. But I guess that could be accomplished by removing all build.commans from the binaries so that spin build just did the validation...
Anyway sorry I am thinking out loud. None of this is actually part of this issue. It just tied into a bunch of stuff that seems in the air about the scope of spin build and its role in the application lifecycle and relationship to spin up and its cronies, so I am kinda brain dumping. Sorry.
Thanks @itowlson , this is all great context. Given that, if we were to invest in this, I would encourage us to think more holistically about a general validation step that occurs at build time. I'd be cautious to simply implement a compose-and-throw-away solution.
Oddly enough I notice when picking up the target worlds stuff that we have to do this for that, although currently we discard the results. So it seems we are already straying into this troublesome "what is spin build" territory! I hadn't really got my head around that implication of the target worlds work... target worlds does fit into what you describe as a "general validation step," although I think @tschneidereit wants it to do more in terms of driving the build process (e.g. flowing into SDKs)... urgh.