[PAL/Linux-SGX] AEX-Notify 3/5: Add AEX-Notify enabling code

[dimakuv]

Description of the changes

Part 3 in AEX-Notify series.

This is a preparatory PR for AEX-Notify support. This PR:

• Introduces the sgx.experimental_enable_aex_notify manifest option.
• Adds architectural flags/bits for SECS, TCS, SSA data structures.
• Adds a Gramine startup check whether AEX-Notify hardware feature is supported by the platform.
• Adds dynamic enablement/disablement of AEX-Notify feature per enclave thread (enable on thread creation, disable on thread termination).

See also related PRs and discussions:

• [PAL/Linux-SGX] Add AEX-Notify support #1530
• [PAL/Linux-SGX] Add AEX-Notify support in exception handling flow #1531
• SGX signal handling flows #1948

How to test this PR?

CI is enough to test that old flows are not broken.

Testing new manifest option sgx.experimental_enable_aex_notify manually:

• Add sgx.experimental_enable_aex_notify = true to HelloWorld CI example
• Test on different platforms

Platform with no AEX-Notify hardware feature

gramine/CI-Examples/helloworld$ gramine-sgx helloworld

Gramine is starting. Parsing TOML manifest file, this may take some time...
error: AEX-Notify hardware feature is not supported.
error: Cannot enable AEX-Notify on this platform (hardware doesn't support it)
error: Parsing manifest failed
error: load_enclave() failed with error: Invalid argument (EINVAL)

Platform with AEX-Notify hardware feature but old Linux

(Linux kernel received AEX-Notify support in v6.2. Below I'm testing with Linux v5.15.)

gramine/CI-Examples/helloworld$ gramine-sgx helloworld

Gramine is starting. Parsing TOML manifest file, this may take some time...
error: Enclave initialization IOCTL failed: Permission denied (EACCES)
error: Initializing enclave failed: Permission denied (EACCES)
error: load_enclave() failed with error: Permission denied (EACCES)

If you wonder why Linux v5.15 enclave init fails with EACCES, check these sources:

• https://elixir.bootlin.com/linux/v5.15/source/arch/x86/kernel/cpu/sgx/ioctl.c#L508
• https://elixir.bootlin.com/linux/v5.15/source/arch/x86/kernel/cpu/sgx/ioctl.c#L113

And compare with Linux v6.2:

• https://elixir.bootlin.com/linux/v6.2/source/arch/x86/kernel/cpu/sgx/ioctl.c#L497
• https://elixir.bootlin.com/linux/v6.2/source/arch/x86/kernel/cpu/sgx/ioctl.c#L114
• https://elixir.bootlin.com/linux/v6.2/source/arch/x86/include/asm/sgx.h#L141

Platform with AEX-Notify hardware feature and new Linux

No changes from the master branch, as AEX-Notify is detected and on-enclave-startup enabled, but not dynamically enabled for enclave threads (see the #ifdef in the code).

gramine/CI-Examples/helloworld$ gramine-sgx helloworld

Gramine is starting. Parsing TOML manifest file, this may take some time...
...
Hello, world

---

This change is 

[dimakuv]

Reviewable status: 0 of 14 files reviewed, 1 unresolved discussion, not enough approvals from maintainers (2 more required), not enough approvals from different teams (1 more required, approved so far: Intel)

---

pal/src/host/linux-sgx/pal_exception.c line 39 at r1 (raw file):

     *        in-enclave runtime doesn't yet implement AEX-Notify flows.
     */
    GET_ENCLAVE_TCB(gpr)->aexnotify = 1U;

Need to add an explanatory comment that AEX-Notify is enabled only for SSA[0] (regular context), and is always disabled for SSA[1] (stage-1 signal handling context).

The disablement of AEX-Notify for SSA[1] is implicit (that particular bit in the SSA's GPR region is by default zero), so we need to at least add a comment explaining this.

And this disablement is important for the stage-1 signal handling flows, which happen in enclave_entry.S. So for future developers, need to add such a comment.

[dimakuv]

Reviewable status: 0 of 14 files reviewed, 1 unresolved discussion, not enough approvals from maintainers (1 more required), not enough approvals from different teams (1 more required, approved so far: Intel)

a discussion (no related file):
Must be applied on top of #2032. Blocking.

---

---

pal/src/host/linux-sgx/pal_exception.c line 39 at r1 (raw file):

Previously, dimakuv (Dmitrii Kuvaiskii) wrote…

Need to add an explanatory comment that AEX-Notify is enabled only for SSA[0] (regular context), and is always disabled for SSA[1] (stage-1 signal handling context).

The disablement of AEX-Notify for SSA[1] is implicit (that particular bit in the SSA's GPR region is by default zero), so we need to at least add a comment explaining this.

And this disablement is important for the stage-1 signal handling flows, which happen in enclave_entry.S. So for future developers, need to add such a comment.

Done