Question on virtual (guest) platforms

[vlsunil]

Does this spec apply for virtual (guest) platforms as well or only host? IIUC, distros will have same kernel working on host and guest. However, the way server-soc requirements exist today, it may not be possible to implement every HW requirement in the guest platforms.

[andreiw]

We should discuss.... IMHO there is a very clear benefit to standardizing VM platforms, and you're right that we can make it challenging to certify if we are not careful with requirements. For example, requiring IOMMU/virtualization inside a VM. On Arm side the desire to bring SystemReady to VMs resulted in the "VE" certification.

Can you provide examples of server soc requirements that you see as being problematic?

[andreiw]

Server SoC IIC_030 is one such requirement (MUST support at least 5 VS-mode interrupt files).

[andreiw]

Server SoC IIC_060 is another (more VS-mode reqs)

[andreiw]

Server SoC IOM_020 mandates IOMMU (should be optional for VMs)

[andreiw]

Server SoC ACS/ADR requirements could be made optional for VMs.

[ved-rivos]

The server SoC specification does not apply to a VM environment. How a virtual machine is implemented is an OS/administrator choice. While there is no aspect of the server SoC that cannot be emulated for VM by a VM - such emulation is typically not required. Some capabilities like ACS/ADR may not even make sense for a VM as they are root-complex/root-port configurations that are typically not passed through to the VM. The OS is free to emulate an IOMMU or more efficient models such as coIOMMU. To support nested virtualization, if an emulated H extension is made available to the VM, the VMM may choose to emulate multiple VS-mode interrupt files and the ISA provides the mechanism to do such emulation.

[andreiw]

The basic requirement for a VM environment is that it can boot the same OSes that boot on real hardware, i.e. follow the single binary distribution model. An x86 VM looks like an x86 host - the OS/admin can't affect how the "virtual chipset" or "virtual firmware" works beyond a very constrained set of choices. Same goes for AArch64, e.g. see Arm hypervisors claiming SystemReady VE certification (e.g. SystemReady SR + VE or SystemReady ES + VE). It is desired to not just model the VMs along the lines of real hardware, but also to certify such environments, which is usually driven by OS vendor requirements.

[andreiw]

So from this POV, if I have a hypervisor that is certified to "server platform 1.0", I would expect the VM environments to model something that looks like a "server platform" environment as well. This would certainly be a reduced environment compared to a physical host, but it's not a free-for-all.

[ved-rivos]

The standardization of the virtual machine execution environment should be in the form of standardization of the BRS for the virtual machine OS. Definitely a x86 VM looks like a x86 machine - but it does not need to look identical to the x86 host machine. In some cases the virtual machine may be provided a subset of even the ISA - for instance to support migration across machines the VMM may create a common denominator ISA among the virtual machines. For instance SBSA requires that all DMA requestors must be governed by an IOMMU - RS_L7SM_01. This does not mandate that a virtual machine be provided an identical SMMU. It may provide a virtual IOMMU - an emulated IOMMU or a paravirtualized IOMMU - to the VM if the VM is enabled to use first stage page tables. Whether nested virtualization is supported in a VM is also an option for the VM and further determines which flavor of virtual IOMMU is made available to a VM. The VE certified instances such as the Google TAU T2A and Oracle OCI A1 compute instances are good examples.

[andreiw]

You're spot on but VE certification implies the VMs are BSA compliant, not just BBR compliance. This could either be ES + VE or SR + VE, but there is the aspect of the virtual SoC being compliant to something. Like, a VM doesn't have to provide an SMMU at all, whereas with our Server SoC spec even a VM would have to provide an IOMMU. I think you're saying that this VM spec could be a separate spec, whereas what I was trying to get to is allowing a subset of the server SoC spec so long as you're in a VM (to avoid this other spec).

[andreiw]

Good topic for tomorrow...

[andreiw]

Actually later today...

[ved-rivos]

I am not saying its a separate spec. It could be a table in the server platform spec to state what is mandatory for virtual machines - for example, the server platform spec states H extension is mandatory but for a virtual machine it would be optional to support nested virtualization. Another example would be performance monitoring counters - a number of hypervisors may not pass through performance monitoring events that provide platform level visibility to a virtual machine since a virtual machine should not observe the physical machine.