Project idea: strongly typed file descriptor integers and deduplication of various checks around them

[oli-obk]

This is less of a fundamental/large-in-behavior change and more a very bitrotty change with possibly not-really-deduplicating-that-much changes that hopefully still make the code more manageable.

Problem statement

Currently we represent file descriptors just as i32 within miri. We mix them with various negative integer values for signalling errors, just like libc does. We also duplicate a lot of error handling logic like

let Some(fd) = this.machine.fds.some_getter(fd) else {
   return this.fd_not_found();
};

and

Ok(Scalar::from_i32(this.try_unwrap_io_result(result)?))

Solution parts

0. consistently use Scalar instead of i32 as the return type of foreign function handlers (groundwork for the following, as we inconsistently use i32 and Scalar. Even using Scalar when i32 would definitely suffice, not just when it could be a pointer).
1. wrap all i32 known-to-be-file-descriptors in a newtype struct together with some convenience functions for reading them and getting the integer back out (into a scalar)
2. return io::Result<FileDescriptionRef> (or the reference versions) from the various getters, instead of Option<FileDescriptionRef>. Forward that where useful, avoiding explicit -1 returns paired with set_last_error, instead handling it with try_unwrap_io_result
3. add get_mut, get, and dup methods to FdTable that return io::Result instead of Option, even though the only io error they ever return is NotFound, allowing us to ? away their result

[RalfJung]

It would also be good to be able to write something like return this.set_last_error_and_return_neg1(...); which would set the last error and then return -1, as that's such a common pattern.

EDIT: Oh, that's what try_unwrap_io_result does. I don't think we are using it everywhere we could, and maybe the name could be improved. It also requires an io::Result<T> but often we want to set EINVAL or some other named constant from libc.

[oli-obk]

These constants (almost?) always have a corresponding ErrKind, so I think we can transform them to io::Result usage. I mean, that is how we convert io::Error to an error value to set, we eval the corresponding constant

[RalfJung]

It seems silly to construct a Result if we hard-code the Err variant - at least we want a method that takes an io::Errror. But I am not sure we can always construct an ErrorKind.

[oli-obk]

Separate idea: instead of hardcoding the io::Result to integer conversion logic, we add a new enum around io::Error that has variants for the extra cases we need. Then we use that enum as the error type everywhere

[noahmbright]

What's the status of this issue? A good chunk of this is taken care of - I could look into fixing any additional pieces.

[tiif]

What's the status of this issue? A good chunk of this is taken care of - I could look into fixing any additional pieces.

It seems like all steps other than step 0 are not implemented yet.

[RalfJung]

I am not convinced that using io::Result everywhere will actually make things cleaner, so I think this is something where we may want to experiment. Not sure what @oli-obk's position is after the other recent error cleanup landed.

We also have way fewer FD numbers floating around now, so a newtype seems less critical -- but could still be a bit of extra type safety. Or it might just be annoying overhead. Hard to say without trying it. :)