proposal-compartments
proposal-compartments copied to clipboard
tc39
Distinct usecases from realms
Hi, I am wondering about what kind of user stories we are thinking about here, and how they differ from realms. This seems like a more light weight solution for things like testing and tooling, but these were also strong arguments for realms. Could you clearly distinguish the unique roles that these two proposals have somehow?
Hiya! So, Realms introduce an entirely new set of global intrinsics, which are likely not what people want in all situations.
For some cases like : limiting the access to importable resource https://github.com/denoland/deno/issues/2768 , or in the case of hot reloading like https://github.com/nodejs/node/issues/49442 creating a new Realm would actually be harmful.
A new Realm would have distinctly different Arrays, Objects, etc. with different prototypes and fail instanceof checks etc. We actually saw a major problem with a case similar to this when parts of the GraphQL ecosystem shipped 2 versions of their libraries 1 ESM and 1 CJS to npm and this kind of distinct difference broke expectations and usage. There were several libraries that faced this problem and all ended up reverting to only ship 1 form of the library (generally the CJS form).
A compartment as has been previously talked about, is purely a manipulation of host behaviors which does not actually cause a "identity discontinuity". That is the important difference.
If you were only able to abstract host behaviors using Realms, you would not be able to address a variety of use cases where identity discontinuity is problematic. For ongoing efforts in this space, test runners like Mocha have tried to create entirely isolated workers for doing things like ESM cache invalidation (by destroying the entire Realm) but decided to back out of that approach due to it not working with the existing expectations of running in the same Realm as the test harness.
Therefore, this difference seems important to keep as a separate API than Realms, and hence the extraction of Realms from SES and Compartments from SES are separate.
great. It is a little hard to wrap my head around both of these proposals and their implications. I were to massively simplify -- and use the list of use cases from the realms proposal, would this cover the following cases? or more or less of them?
- [x] Web-based IDEs or any kind of 3rd party code execution uses same origin evaluation.
- [ ] Fiddler & Co.
- [x] ? JSPerf & Co.
- [x] Test frameworks (in-browser tests, but also in node using vm).
- [x] testing/mocking (e.g., jsdom)
- [x] Most plugin mechanism for the web (e.g., spreadsheet functions).
- [ ] Sandboxing (e.g.: Oasis Project)
- [ ] ? Server side rendering (to avoid collision and data leakage)
- [x] in-browser code editors
- [x] in-browser transpilation
When you say "plugin mechanisms", i think that's a broader category that might include react and other components?
@codehag likely there is overlap to a lot of these, but they differ in actual usages on if you would want both or just 1. The answer likely is things want to be able to do both behaviors usually but in a 1 Realm to many Compartments potentially with some number of 1 Realm to 1 Compartment usages as well.
- [x] Web-based IDEs or any kind of 3rd party code execution uses same origin evaluation.
Likely this would only use compartments and not realms if the intrinsics are safe to share (freeze them and remove action at a distance vectors [roughly what SES tries to do]). However, it isn't possible to completely "virtualize" all host behavior purely by creating new Realms due to import() and import.meta. See previous TC39 meeting notes about problems virtualizing these behaviors and a request for ensuring hooks to virtualize in the future. However, if you use Realms, then problems with identity discontinuity exist so you probably want both in reality.
- [ ] Fiddler & Co.
Likely since Fiddlers are running as a sandbox, but not having multi-tenancy or any sort, purely having Realms might work, but lack of Web APIs would be problematic. Keep using the existing <iframe> behavior and maybe a Realm inside to separate the harness from the user code. I don't see Compartments being of much direct use here for the isolation. For actual things like external assets and/or visualizing other locales (we had a really bad bug with this in French Canadian at a previous job)/times/etc. compartments may be useful, but to my knowledge no fiddle style editor actually supports those things currently.
- [x] ? JSPerf & Co.
Likely Realms are all that is desired here since they are trying to isolate the code, but not change host behaviors. This is a little bit misleading though since if they wanted to manipulate how import works or other things during benchmarking they would want Compartments. To my knowledge lack of configurability has led to no existing testing across those host differences.
- [x] Test frameworks (in-browser tests, but also in node using vm).
Likely they want Compartments more than Realms if they are already using vm.* in Node, but only because of lack of Realms. Compartments would allow manipulation of imports to align with testing utility and with the locale / time testings mentioned above. Due to a lot of these frameworks needing to share intrinsics between the harness and the tests I would state that while Realms do give them isolation they want, to some extent host specific APIs such as same origin <iframe>s / vm can alleviate their needs as current status quo shows.
- [x] testing/mocking (e.g., jsdom)
Likely they want both for the same reason as test frameworks. In particular, mocking really wants to be able to have multiple imports of the same target to potentially resolve to different modules.
- [x] Most plugin mechanism for the web (e.g., spreadsheet functions).
You likely want both, in particular to censor host behaviors and ban importing arbitrary things which can be used for tracking purposes and/or can load problematic code (this does not stop all vectors). Even if you isolate an address space with the status quo techniques, you still want to prevent some host behaviors (to some extent CSP does allow this in the Web in a non-programmatic fashion).
- [ ] Sandboxing (e.g.: Oasis Project)
https://github.com/tildeio/oasis.js ? I am less familiar with that project than current SES / Lightning Web Components / etc.
It might be useful with Realms, but it seems it does use some <iframe> specific APIs with their docs about non-Shared DOM that I am unsure they would be able to completely use a less host specific API. In theory they could use these proposals but currently are only using Realms with additional host APIs effectively.
- [ ] ? Server side rendering (to avoid collision and data leakage)
For ahead of time rendering absolutely since it would isolate things without worries. Would want to see benchmarks for on-demand rendering. We have had a few minor bugs from Isolation problems in production but only a handful over a few years at 1200 builds per deploy at work. For us, the bigger thing we run all our builds in a locale matrix, and being able to properly interact with such a thing in an isolated way is what we want. We don't really need the address space isolation given our work experience and probably don't want it so that our render cache can be shared during all the builds..
- [x] in-browser code editors
I am unsure if this needs Compartments, I don't see a need to isolate the host behaviors unless the editor is trying to emulate things that differ from the current page. They could want to emulate things like import maps, partial evaluation, a REPL etc. So if they want those things, they likely want a Realm, but I don't see the direct need for compartments outside of specific features that might not apply to all editors. Things like the TypeScript compiler can run without either of these proposals which likely is more important.
- [x] in-browser transpilation
They both could be used for this. In particular being able to create a way to instrument code is not supported in browsers via intercepting import in any way. Compartments would provide an opt-in way of taking a perf hit to do the transpilation without affecting the render thread defaults. I have done tests using service workers and compatibility with Node's runtime loader APIs and it ends up with up to 4 fetches per transpilation using service workers. Likely, the goal of anything wishing to do transpilation will want to do it off thread and Node is also moving to this model so they may just delegate to a Worker inside the hooks.
@codehag , think of compartments as further subdivisions within a single realm. The similarities between both realms and compartment are indeed confusing. Here is a quick way to tell them apart:
| feature | iframe & realm | compartments |
|---|---|---|
| Global object | per realm | per compartment |
| Global lexical scope | per realm | per compartment |
| Intrinsics | per realm | shared between all compartments of the same realm |
You can also see compartments as an extra construct on the scope chain, which is typically (but not always):
realm > compartment > module > class > function > block
