Plugins V1

[thedodd]

As I've been hacking on a POC of the WASM+WASI plugin system, it seems quite clear that the level of effort is still quite high, and WASI and other standards (WIT comes to mind), especially other parts of the WASI protocol such as the sockets API, are not stable at all yet, and especially not in a reasonably useable state. Let's punt on the WASM+WASI plugin system for a future v2.

In the immediate term, it makes sense to build upon what has been introduced in #221 (the hooks system). Not as a replacement, but as a sibling pattern. A history of much more discussion on this topic can be found in #104.

Architecture

• Every plugin is simply a binary which can be downloaded by Trunk. We can define standards around how plugins should be archived for Trunk to be able to unpack them properly, along with how to include additional files and the like.
• Plugins are invoked by trunk and are passed a protobuf payload on stdin. All plugin behavior should be based upon the protobuf payload.
• Plugins are expected to return a protobuf payload on stdout. Stderr can be used by the plugin for diagnostics.
• Plugin output is expected to declare the name of any files created, deleted &c.

Declaring Plugins

• Plugins are declared in the Trunk.toml under a [[plugin]] section.
• The location of the plugin on the internet must be declared. We'll do some exploration on how to support private repos/artifacts.
• The "rel" type to be used in the index.html to refer to this plugin for use.

Using Plugins

• Plugins are declared for use the same way that other Trunk built-in pipelines are, and their "rel" type must match its declaration in the Trunk.toml.
• Attributes declared on the plugin's tag in the index.html will be passed to the plugin as part of the protobuf payload when invoked.
• Plugins can be executed (for now) in one of three stages: pre-build, build, post-build. This primarily governs the files which will be visible to the plugin. EG, plugins invoked in the post-build stage will have visibility of all files produces by the built-in pipelines as well as the output of other build stage plugins.
• We could also allow plugins to declare custom stages, as well as a dependencies comma separated list of other stage names. Such a stage would only be executed after all of its dependencies have been completed. Basic workflow orchestration.

Misc

• This will be an opportunity for us to build the "manifest" system, which could be used by #9 .
• Other folks have requested a manifest for other uses. Such uses could probably just be plugins.
• The current built-in pipelines will be adapted (pretty minimal) to operate within this framework (just not protobuf or external calls needed in the built-in pipelines case).

[lukechu10]

How will this work with different platforms? Is trunk expected to install the correct binary for the platform it is currently running on? What if there isn't a binary for the platform? I like the simplicity of this design but I'm afraid there will be problems across platforms (especially for Windows and Macs with M1 chips)

Possibly related to #212

[thedodd]

@lukechu10 the standard should be that a plugin should provide a binary for the same platforms as Trunk. Any "official" plugins which are part of the Trunk org will adhere to the same standard.

If the plugin does not offer binaries for some OS/Arch, but it can be compiled on that arch, then we should have a mechanism to allow for the plugin to be "discovered" as a system installation, which is exactly what the current download system does for wasm-opt & wasm-bindgen.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.