Consider allowing user-defined structs to be typed as TYP_SIMD

[tannergooding]

Currently the JIT has various specialized handling for TYP_SIMD8, TYP_SIMD12, TYP_SIMD16, and TYP_SIMD32. However, this specialized handling is limited to the framework defined Vector2, Vector3, Vector4, Vector<T>, Vector64<T>, Vector128<T>, and Vector256`.

In various cases and for various reasons, users may define their own types rather than using the framework types. This typically occurs for math and game frameworks where they are defining equivalents to Vector2/3/4.

As such, it may be beneficial for the JIT to recognize such structs and treat them as the appropriate TYP_SIMD. I would propose we initially limit this to TYP_FLOAT and can consider expanding in the future as appropriate.

category:design
theme:vector-codegen
skill-level:expert
cost:large
impact:medium

[tannergooding]

This was brought up by @saucecontrol here: https://github.com/dotnet/corefx/issues/36379#issuecomment-545105766 and has been raised on a couple of different other threads in the past.

[tannergooding]

CC. @CarolEidt for thoughts

[tannergooding]

System.Drawing, Windows Forms, and WPF would all likely benefit as they have various low-level structs defined that use 2-4 float elements (WPF also has a fair number using TYP_DOUBLE). There are also several System.Numerics and ML.NET types that may benefit.

[CarolEidt]

Could you articulate the scenario(s) and expected benefit? I don't foresee auto-vectorization in the JIT any time soon, and other than that I'm not sure what value this would provide. I'm hoping that we can eliminate, or at least seriously reduce, the abstraction penalty for wrapping the SIMD types, but I gather that wouldn't address this issue.

[john-h-k]

@CarolEidt From what I see, it could be useful if also supported with a new intrinsic overload of Vector{nnn}.Create, where such types would not need to be loaded from memory. E.g

Foo x = ...;
Vector128<float> z = Vector128.Create(x);

where Foo is a TYP_FLOAT, the JIT could elide any moving around if it has been promoted to a SIMD register

[CarolEidt]

@john-h-k - this is a scenario that is (theoretically) already covered - that is, the JIT should have already put x into a floating point register, such that the Create intrinsic should never have to load it from memory. If there are cases where that's not happening, it would be great to see an example.

[john-h-k]

Surely not if Foo is

struct Foo { public float X, Y, Z, W; }

I didn't think it put that (entire) struct in a FP register?

[tannergooding]

The primary benefit would be zero/low cost conversion between the built-in types and user-defined types

That is, I might define the following struct

readonly struct Color {
    public float Red { get; }
    public float Green { get; }
    public float Blue { get; }
}

Today, in order to get a Vector3 "efficiently" you need to either implement it internally as a Vector3 or you need to use Unsafe.As . Representing it as Vector3 may be undesirable because Vector3 is mutable and can lead to undesirable copies for a readonly struct. Using Unsafe.As is also not a nop and involves spilling values to the stack and then reloading them as a Vector3. Getting a Vector3 is an important scenario as it allows you to then get a Vector128<float> so you can intrinsify your implementation.

If we were to track them as TYP_SIMD12 instead, then we could trivially special-case the conversion of TYP_SIMD12 to TYP_SIMD12 (similarly to how we do Vector4 to Vectore128<float>).

There are also other minor benefits to doing this.

• More efficient stack spilling. These user defined types would be spilled to/read from the stack as TYP_SIMD16, making it cheaper to work with.
• Easier ABI handling/checking. Since these types have special ABI requirements on Linux and Windows __vectorcall, they need to be tracked as HFAs. Tracking them as TYP_SIMD in the first place should simplify that logic as we then know any TYP_SIMD with a base type of TYP_FLOAT or TYP_DOUBLE is an HFA (except for Vector128<float/double> which is __m128/__m128d).
• Easier auto-vectorization support in the future. These types that qualify as TYP_SIMD are exactly the types we may want to auto-vectorize. Having them tagged as TYP_SIMD should make it simpler to identify types which would benefit from auto-vectorization (even if it is just simplified support for scenarios where you perform the same operation on each element, in order which would allow most operator + to become addps).

[mikedn]

It may have been better if the intrinsic support wasn't designed around well known types and methods but rather around attributes, layouts and signatures. For example:

[Intrinsic(IntrinsicId.Float128)]
struct Color {
    public float R, G, B, A;
}
static class ColorOps {
    [Intrinsic(IntrinsicId.Float128Add)]
    public static Color Add(Color c1, Color c2) {
        throw new NotSupportedException();
    }
}

The developers wouldn't be forced to use the particular types and method the runtime provides or be forced to pay the potential overhead of building an abstraction layer.

Bonus - if you can get the VM to handle the intrinsic ID storage and retrieval in an optimal manner (I suspect that there may be issues with it) you avoid the name based lookup throughput issue.

But if this has not been done from day 1 it's probably too late to do something like this now.

[CarolEidt]

@john-h-k - It sounds like you intended your example to mean that Foo was a struct containing multiple floats, in which case it seems like it's the one that Tanner describes.

@mikedn - I think you're right that what you describe would have been a better way to go, and perhaps we can work toward a model that is at least closer to that in usability.

I'm still unconvinced that, from a JIT perspective, we want to recognize these explicitly as SIMD types. Rather, it seems like an argument for improving the JIT's ability to allocate entire structs to SIMD registers if it is profitable. Note that even with today's fixed-size vector types (Vector2, Vector3, and Vector4) the JIT uses heuristics to determine whether it is profitable to enregister the whole thing, or to separately enregister the individual fields. While these heuristics leave room for improvement, it seems like a similar overall approach to structs that fit in a SIMD register would be the best way to go.

Easier ABI handling/checking. Since these types have special ABI requirements on Linux and Windows __vectorcall, they need to be tracked as HFAs. Tracking them as TYP_SIMD in the first place should simplify that logic

I agree that it may be worthwhile to consider utilizing the SIMD types in the ABI handling, though I would caution that it might actually complicate the ABI handling, since not all of the HVA/HFA types have corresponding SIMD types, and not all ABIs handle these types specially. I think further investigation and/or experimentation would be needed.