Use async pid fd instead of blocking waitid to wait for a child process to exit

[jprendes]

This PR replaces the blocking waitid call with an async implementation based on pid fd.
There's a race condition where containerd-shim reaps child processes.
If the child process has already been reaped, query containerd-shim to get the process status.

Note that this race condition is already present in the current implementation, but runwasi's waitid is very likely to win the race, the introduction of async evens out the odds between runwasi and containerd-shim.

This PR is in preparation to move the whole shim implementation to async.

[andreiltd]

LGTM!

[Mossaka]

LGTM!

[jprendes]

I still need to figure out why CI is failing. I think the http_poxy container is not receiving the SiGINT for some reason. But I don't see how that can relate to this. I'll keep digging tomorrow.

[Mossaka]

Great! Thanks for working on this

What I'd like to see, perhaps as a follow up, is some unit tests for edge cases like

1. What would happen if the pid does not exist
2. what would happen if the orignal PID is reused by a new process after the first one exists.
3. what would happen to call wait() when containerd-shim has already reaped the process

[Mossaka]

Great! Thanks for working on this

What I'd like to see, perhaps as a follow up, is some unit tests for edge cases like

1. What would happen if the pid does not exist
2. what would happen if the orignal PID is reused by a new process after the first one exists.
3. what would happen to call wait() when containerd-shim has already reaped the process

[Mossaka]

Note that this race condition is already present in the current implementation, but runwasi's waitid is very likely to win the race, the introduction of async evens out the odds between runwasi and containerd-shim.

Just to clarify, in this PR, you did not resolve this issue, right? If yes, could you please raise an issue against the repo so that we can track this one.

[jprendes]

Just to clarify, in this PR, you did not resolve this issue, right?

It doesn't resolve the underlying race condition, but it covers all possible outcomes, so it's not an issue for runwasi.

could you please raise an issue against the repo so that we can track this one

To be fair, we could just use the containerd_shim reaper alone, and we wouldn't need the pidfd part.
I think that's how containerd_shim intendeds it to be used.
However, that breaks our tests, as we don't bootstrap the containerd_shim reaper in tests.
If we do bootstrap the containerd_shim reaper in tests, then we would break tests that use process::Command, as they would hit the same race condition.

[jprendes]

1. What would happen if the pid does not exist

During wait, it would hit the ECHILD branch, and then wait forever in the c8d_shim reaper. Fixed it so that we wait with a timeout.

2. what would happen if the orignal PID is reused by a new process after the first one exists.

That's not a problem.

• PIDFD is resilient to that by using a dedicated FD instead of the PID.
• The reaper should send the correct event regardless. Assuming the reaper always wins, that would mean that the reaper would broadcast two events with the same PID, but we stop listening after the first hit.

3. what would happen to call wait() when containerd-shim has already reaped the process

As long as the struct is created before the process exits, that's not a problem. That's why we create the struct BEFORE calling container.start(). I added a comment explaining this in the code.

[Mossaka]

LGTM!

The merge-base changed after approval.