Compare and Contrast against ShadowRealms

[mgaudet]

I find myself trying to figure out why we would want both this proposal and https://github.com/tc39/proposal-shadowrealm

There seems to be a lot of overlap here between ShadowRealms and this proposal and it's clearly (to me) a conceptual competitor.

Can you add some information about how this differs, why one would use one or the other?

[naugtur]

Thank you for this question, I'll make sure we highlight how this differs from shadow realm in the slides.

[kriskowal]

For the record, this differs from ShadowRealms in a few ways.

1. new globalThis.Global() creates a new global with all the same internal slots as globalThis. We reason, and seek your confirmation, that this is mechanically less risky than the new category of global in ShadowRealm.

2. The new global shares the same Realm-scope intrinsics as the globalThis, except for evaluator intrinsics like eval that must be bound to the new global.

The calculus of trade-offs is also different.

We expect such globals are much cheaper to instantiate.

With new Global, it’s also much more easy for platform code, like the Node.js runtime, to stand on one global and module map and for the underlying application to stand on a different global and module map, but be able to freely recognize each other’s objects, e.g., [] instanceof otherGlobal.Array.

Some of our motivating use cases are similar, like confinement of unreliable code, like test cases and AI slop. But ShadowRealms have their own batch of intrinsics, which can be useful for confinement when the adversary is expected to mutate the intrinsics.

Consequently, confinement with new Global pairs well with frozen shared intrinsics.

ShadowRealm:

• Good for testing shims
• Good for confining slop that might need to be able to foul its own bed
• Slow to start
• High heap footprint
• Slow message passing
• Not sensitive to the override mistake

Globals and Frozen intrinsics:

• Good for testing modules, mock modules
• Good for confining slop that can fail if it attempts to foul the shared intrinsics
• Fast to start
• Small footprint
• Fast message passing
• Shared intrinsics
• More sensitive about override mistake

Some of the motivating cases, like DSLs (consider Jasmine, Jest, Gulp, configuration files), we feel stand on their own even without frozen intrinsics.

Of course, they stack, if you need defense in depth. The champions would like to see both succeed, but will not hesitate to exploit the ways we hope New Global is less troublesome for VM implementors than ShadowRealm.

[bakkot]

The presentation today was heavily focused on security use cases (supply chain attack mitigation, etc).

It seems to me that this was also the primary motivation for ShadowRealm, and in particular the decisions to not automatically copy "powerful" globals and to have the callable boundary were (as I recall) motivated by security considerations.

If this proposal is adequate to address those use cases, I don't understand why we need ShadowRealm. If it's not, I don't think it makes sense to consider security as a motivation for this proposal (but then am unclear on the motivation for this proposal - the remaining use case from the above-linked presentation is "testing infrastructure", which doesn't seem enough to warrant all this machinery).

Edit: perhaps this would be better phrased as, under what threat models are compartments necessary and sufficient, and under what threat models are compartments not sufficient and ShadowRealms necessary and sufficient?

[kriskowal]

Another non-final take on this question:

If you’re creating a plugin system for third-party extensions, Compartments are attractive because they are light-weight. Compartments remain suitable until a plugin needs the ability to mutate intrinsics, at which point it needs a ShadowRealm of its own that is not shared with other plugins. A ShadowRealm is suitable until the plugin needs high-resolution timers, at which point each plugin needs its own process. The process in turn needs a ShadowRealm so the plugin can’t escalate/escape.

I think ShadowRealm is equivalent to the confinement afforded by a Wasm instance with a full JavaScript interpreter, but much more economical.

For supply-chain attack mitigation, Compartments are attractive because most applications work without modification if every package gets its own Compartment. Putting every package in separate realms or processes is simply not tractable.

So, a ShadowRealm cannot stand in for a Compartment if the motivating case is confining packages. A Compartment is not a substitute for ShadowRealm if a plugin needs its own, mutable batch of intrinsics.

In platforms with plugin systems (Salesforce, Figma, Agoric, MetaMask, &c), it is possible to have both of these concerns come to bear simultaneously. A plugin can be confined once in order to protect the platform, and a plugin can confine its own dependencies to defend itself. And, different plugins may require different kinds of confinement.

[bakkot]

That answer seems to be phrased purely in terms of what kind of code can feasibly/performantly execute in a Compartment or ShadowRealm. Does that mean that these are aimed at addressing precisely the same threat models, and the difference is just about whether the code needs mutable globals?

Separately, I don't understand how a Compartment can feasibly be used for confining packages. When all globals are copied by default, you have to assume that some of those are going to be powerful (or allow you to reach powerful things). Asking the person making the Compartment to determine the minimal set of globals needed by the code in that Compartment seems infeasible. Limiting the set of reachable globals to a broadly useful but non-powerful subset is (as I understand it) much of the point of ShadowRealms. And of course the code in the Compartment can trivially mutate values passed to it from other modules, and in many cases that mutation would be sufficient to escape the Compartment. Even with a hypothetical lockdown mode you can't assume that user-authored JS code is going to start freezing all of the objects it produces. Avoiding this risk is (as I understand it) the point of the callable boundary in ShadowRealms.

Also, "this transitive dependency updated to include a polyfill for a new feature and therefore now incidentally assumes that globals are mutable" is the sort of change which people are going to be making in patch releases. If that kind of change requires refactoring your entire stack to switch from a Compartment to a ShadowRealm I don't think there's going to be much appetite for using this for dependencies.

[kriskowal]

If I may answer incrementally, the key insight of LavaMoat is that it creates a policy for the compartment map based on heuristic static analysis. This is the ToFU (trust on first-use) we mentioned in the presentation today. Which is interesting, because it obligates code that is reaching for authority to make obvious that they are reaching for authority, or they will not by default get it.

https://github.com/lavamoat/lavamoat

[kriskowal]

That answer seems to be phrased purely in terms of what kind of code can feasibly/performantly execute in a Compartment or ShadowRealm. Does that mean that these are aimed at addressing precisely the same threat models, and the difference is just about whether the code needs mutable globals?

Speaking only to the case of Compartment when used in conjunction with a usermode Lockdown, I think this is a fair characterization, but I’ll let others with a better grasp on threat-modeling speak to threat-modelling. It would not be fair to imply that addressing this threat model is the only use case we are bringing. Just, an important one to the champions.

Separately, I don't understand how a Compartment can feasibly be used for confining packages. When all globals are copied by default, you have to assume that some of those are going to be powerful (or allow you to reach powerful things).

Note keys. This would be the way we configure a Compartment to only be endowed with powerless, hardened intrinsics. https://github.com/endojs/proposal-module-global/tree/main/previously#interfaces

Asking the person making the Compartment to determine the minimal set of globals needed by the code in that Compartment seems infeasible.

ToFU mentioned above.

And of course the code in the Compartment can trivially mutate values passed to it from other modules, and in many cases that mutation would be sufficient to escape the Compartment. Even with a hypothetical lockdown mode you can't assume that user-authored JS code is going to start freezing all of the objects it produces. Avoiding this risk is (as I understand it) the point of the callable boundary in ShadowRealms.

It is true that the interactions between modules remain porous but in HardenedJS we also provide a harden mechanism that makes it relatively straightforward to defend at the module boundary for the edges worth defending. For some applications, that may be all edges. For supply chain attack mitigation, being able to approach this problem incrementally is tremendous.

Also, "this transitive dependency updated to include a polyfill for a new feature and therefore now incidentally assumes that globals are mutable" is the sort of change which people are going to be making in patch releases. If that kind of change requires refactoring your entire stack to switch from a Compartment to a ShadowRealm I don't think there's going to be much appetite for using this for dependencies.

We find that the appetite in practice largely varies depending on how accountable the application is to its users. You are correct that not many applications are yet even aware that mounting this kind of defense is feasible. And, a lot of applications can be cavalier with PII, for example, because it leaks in generally weakly traceable ways. I take as given that the need to mount this kind of defense is increasing, though.

[mhofman]

under what threat models are compartments necessary and sufficient, and under what threat models are compartments not sufficient and ShadowRealms necessary and sufficient?

I'm not sure phasing it this way makes sense. The way I see it, these are solutions operating at different layers, which can be used for the same purpose. A VM and containers provide a roughly similar analogy. You can use a VM to isolate an application, but in most cases a container is sufficient. Before containers, you would use a VM to isolate a single application, but it was overkill. And of course, VM still exist because they do solve some use cases that containers don't.

I don't understand how a Compartment can feasibly be used for confining packages.

I would like to point out that both ShadowRealm and Compartment are not meant to be directly used by applications, but are a tool for security solutions to use, with the application hopefully unaware of their presence. They are also not the only tools to be used. What globals are available by default does not matter, as these can be removed or added as necessary by these solutions (I actually argued that ShadowRealm should just offer all the same globals the principal realm does).

As mentioned, a solution like lavamoat uses Compartment to run packages in their own global scope, endowing the necessary global powers discovered and subsequently enshrined in policy. Lavamoat also "repairs" some intrinsics like preventing escape of the compartment's global scope through Function.prototype.constructor. Finally it locks down the repaired shared intrinsics preventing otherwise unconnected packages from indirectly affecting each other.

This all works today to confine packages, with the shim implementation we have of Compartment and lockdown.

"this transitive dependency updated to include a polyfill for a new feature and therefore now incidentally assumes that globals are mutable"

I'm not sure where this is coming from. I do not believe this is representative of any best practices that the JS community has been following for years. Libraries in general do not mutate intrinsics, unless those libraries' purpose are exactly to apply polyfills. Mutating intrinsics to provide the expected environment is in the responsibilities of the application, not nested in transitive dependencies.

As such I do not believe that is a realistic case to account for. You wouldn't switch an application dependency isolation from a solution based on Compartment to one based on ShadowRealm. Again, some application may have use cases better solved by ShadowRealm, like plugins or applets use cases, but for dependency isolation of a single application, I do not believe ShadowRealm is the correct approach, and I do not believe it ever was advertised as such (see explainer).

the code in the Compartment can trivially mutate values passed to it from other modules, and in many cases that mutation would be sufficient to escape the Compartment.

Without defensive programing a package would indeed be able to tamper with the objects shared by other packages. While true, at least Compartment with lockdown provides a foundation on which packages can adopt defensive techniques to protect themselves without having to worry about how compromised the global environment is. That effectively reduces their duties to hardening the objects they expose, and avoid exposing excess authority. We are actually hopeful to propose a standalone harden to let libraries more easily transitively freeze by property walk their exported objects.

All this is describing layers of defenses that can be incrementally adopted. Isolation of dependencies in compartments drastically reduces the blast radius of a drive-by untargeted attack. It does not prevent targeted attacks leveraging access an existing dependency may already have. Most dependencies do not require special powers anyway. Then dependencies that follow encapsulation best practices can make their objects tamperproof, and become resistant to targeted attacks, because they run on a solid foundation.