Add support for `wasi:config/runtime` virtualization

[scothis]

The wasi:config/runtime interface is a way to pass configuration to other modules with a standard interface. Similar to wasi:cli/environment the core data model is a set of key-value pairs.

The proposed API and CLI support in WASI-Virt mirrors the support for environment variables that currently exists.

The config runtime proposal is currently stage 2 and does not a have a release. Because of that, I understand it may not be desirable to land support inside wasi-virt at this time. However, if there is a willingness to move forward, I can refine this PR to DRY-up the similarities between env and config, and flesh out docs, tests, etc.

[guybedford]

Thanks so much for looking into this. Nice to see it implemented as a separate adapter component, to verify - if it's not used, are we able to DCE the adapter similarly to the environment subsystem?

I will aim to review in the next few days.

[scothis]

to verify - if it's not used, are we able to DCE the adapter similarly to the environment subsystem?

Currently following the same pattern used for wasi:cli/environment.

The config adapter is only injected into the module when the config option is enabled.

https://github.com/bytecodealliance/WASI-Virt/pull/82/files#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R156-R158

...and stripped out when not active

https://github.com/bytecodealliance/WASI-Virt/pull/82/files#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R229-R233

Something that needs to be handled better is --allow-all, which adds the wasi:config/runtime import to the resulting module even if the wrapped module doesn't import that interface. Hosts that don't provide the interface (like wasi:http/proxy and wasi:cli/command) will no longer be able to satisfy the module imports.

Have you thought about limiting the imported interfaces on the module to not exceed the imports on the wrapped module? I can't think of a reason to expose an import that will never be consumed.

[guybedford]

Have you thought about limiting the imported interfaces on the module to not exceed the imports on the wrapped module? I can't think of a reason to expose an import that will never be consumed.

This makes a lot of sense - a DCE of the virtual module to the known module it is adapting, when the virtual module is not being generated on its own but in this composition.

Would you be interested in doing that here or are you thinking of a follow-up perhaps?

[scothis]

A separate PR that lands first makes sense. I'll be offline for a few days, but can take a look next week.

[scothis]

Rebased on main and added tests/doc.

[scothis]

confirm it's definitely not being pulled in for other subsystems

I'm exactly sure what you're looking for, so here's a few examples:

no host access

wasi-virt -o virt.wasm

virt.wasm:

package root:component;

world root {
  export wasi:cli/[email protected];
  export wasi:config/[email protected];
  export wasi:cli/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:clocks/[email protected];
  export wasi:clocks/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:filesystem/[email protected];
  export wasi:filesystem/[email protected];
}

stdio only

wasi-virt --allow-stdio -o virt.wasm

virt.wasm:

package root:component;

world root {
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];

  export wasi:cli/[email protected];
  export wasi:config/[email protected];
  export wasi:cli/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:clocks/[email protected];
  export wasi:clocks/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:filesystem/[email protected];
  export wasi:filesystem/[email protected];
}

runtime config only

wasi-virt --allow-config -o virt.wasm

virt.wasm:

package root:component;

world root {
  import wasi:config/[email protected];

  export wasi:cli/[email protected];
  export wasi:config/[email protected];
  export wasi:cli/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:random/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:clocks/[email protected];
  export wasi:clocks/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:filesystem/[email protected];
  export wasi:filesystem/[email protected];
}

all

wasi-virt --allow-all -o virt.wasm

virt.wasm:

package root:component;

world root {
  import wasi:cli/[email protected];
  import wasi:config/[email protected];
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:clocks/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:http/[email protected];
  import wasi:http/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:clocks/[email protected];
  import wasi:filesystem/[email protected];
  import wasi:filesystem/[email protected];

  export wasi:cli/[email protected];
  export wasi:config/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:io/[email protected];
  export wasi:clocks/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:sockets/[email protected];
  export wasi:http/[email protected];
  export wasi:http/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:cli/[email protected];
  export wasi:filesystem/[email protected];
  export wasi:filesystem/[email protected];
}

composed

The same component is wrapped for each of the examples below.

component.wasm:

package root:component;

world root {
  import wasi:logging/logging;
  import wasi:config/[email protected];

  export wasi:cli/[email protected];
}

no host access

wasi-virt -o virt.wasm component.wasm

virt.wasm:

package root:component;

world root {
  import wasi:logging/logging;

  export wasi:cli/[email protected];
}

runtime config only

wasi-virt --allow-config -o virt.wasm component.wasm

virt.wasm:

package root:component;

world root {
  import wasi:logging/logging;
  import wasi:config/[email protected];

  export wasi:cli/[email protected];
}

all

wasi-virt --allow-all -o virt.wasm component.wasm

virt.wasm:

package root:component;

world root {
  import wasi:logging/logging;
  import wasi:config/[email protected];

  export wasi:cli/[email protected];
}

all (without filtering added in #83)

wasi-virt --allow-all -o virt.wasm component.wasm

virt.wasm:

package root:component;

world root {
  import wasi:logging/logging;
  import wasi:cli/[email protected];
  import wasi:config/[email protected];
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:io/[email protected];
  import wasi:clocks/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:sockets/[email protected];
  import wasi:http/[email protected];
  import wasi:http/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:cli/[email protected];
  import wasi:clocks/[email protected];
  import wasi:filesystem/[email protected];
  import wasi:filesystem/[email protected];

  export wasi:cli/[email protected];
}

[scothis]

What if we drop it from --allow-all so that explicit opt-in is required. More options can be introduced in the future.

The interface would still show up on the exports of the virt component, but that should be safe.

[guybedford]

The interface would still show up on the exports of the virt component, but that should be safe.

I believe we don't ever include subsystems that aren't explicitly included, are you sure this would still be the case? Cutting the bytes is important I think when features aren't needed.

[scothis]

I believe we don't ever include subsystems that aren't explicitly included, are you sure this would still be the case?

I followed the pattern from environment. When the subsystem isn't enabled the interface still exists but the functions are replaced.

https://github.com/bytecodealliance/WASI-Virt/blob/c4512614d0bbe50044b93577099232c1701b06e2/src/lib.rs#L273-L278 https://github.com/bytecodealliance/WASI-Virt/blob/c4512614d0bbe50044b93577099232c1701b06e2/src/virt_env.rs#L233-L250

Cutting the bytes is important I think when features aren't needed.

I generated a set of virt components with this branch and from main@c451261. The latent capability adds 3K over the equivlent., actually using --allow-config adds 1K. The delta between --allow-config and --allow-env is -1K. The delta between --allow-config and --allow-all is 88K.

219K branch-allow-all.wasm
131K branch-allow-config.wasm
132K branch-allow-env.wasm
130K branch-none.wasm
216K main-allow-all.wasm
130K main-allow-env.wasm
127K main-none.wasm

[guybedford]

Thanks for confirming - so the exports are still written even when the subsystem is unused, but that isn't an issue for compatibility because exports are always optional for importers. And from a size perspective, we get exactly the same DCE situation as for env. The case where an unused subsystem generates exports are always stub exports anyway and this is the standard approach already, and this PR fully implements strip and stub virt operations to align with that.

Appreciated for sharing the numbers as well to verify the DCE!