SpriteMesh + SpriteMaterial

[PVDoriginal]

Objective

The objective is reworking Sprites to use the Mesh backend instead of the current, severely outdated, one.

The 2D Ecosystem would grately benefit from sprites having access to all the mesh utilities, such as ExtensionMaterials, without having to be updated and maintained separately.

We could allow things such as attaching a shader to a Sprite to give it an outline, which is a very accessible feature in other engines that Bevy currently lacks any streamlined support for (#16170).

This PR notably contributes to #13265, while also completely fixing #15021 and possibly other sprite-related issues.

Solution

I've introduced a new SpriteMesh component. It is an exact replica of Sprite, having the same API, except for an extra alpha_mode field.

The purpose of this is to eventually remove Sprite and all its backend, and rename SpriteMesh to Sprite, which should hopefully be a seamless transition for our end-users.

Internally, SpriteMesh is just an abstraction over adding a Mesh2d and the new SpriteMaterial to an entity, and a user should be able to just do the latter if they prefer, essentially merging the ease-of-use of Sprites and flexibility of Meshes.

In its current state, SpriteMesh is completely usable and produces the same behavior as Sprite. There is however a notable exception:

• The max_corner_scale parameter when slicing now produces different results. Before, in the sprite_slice example, it was set to 0.2, causing the corners to be half of their scale. I am fairly confident this was a bug since I haven't found any correlation between 0.2 and what was being displayed. Instead, the SpriteMesh now requires 0.5 to produce the same result, 0.2 resulting in a much smaller corner (a fifth of its size).

Sprite, max_corner_scale=0.2	SpriteMesh, max_corner_scale=0.2	SpriteMesh, max_corner_scale=0.5

Otherwise, replacing a Sprite with a SpriteMesh should have no visual difference.

There were also a few bugs that I encountered with the Sprites, related to scaling and flipping them a certain way producing
weird behavior such as the sprite completely disappearing. These are not present in the new SpriteMesh.

While the API and visual behavior remain the same, the underlying logic has been completely rewritten:

• No custom RenderApp logic is used anymore, SpriteMesh/SpriteMaterial exclusively uses the API provided by Material2d.

• Slicing and Tiling are now done purely inside the shader through UV mapping and manipulation (all done in constant time). Before, they generated multiple SpriteInstances for each tile / slice, being extremely unmaintainable and adding significant overhead. This fixes Move sprite 9 patches to a shader to avoid thin lines #15021.

Testing

• I mostly ran tests by overlapping Sprites with SpriteMeshes having the exact same configuration and making sure there are no differences in behavior. However, I did essentially have to rework the whole Sprite API to work with Meshes and I can't guarantee that I haven't missed anything.

• I tested the new SpriteMeshes in bevymark. I'd say there's an ~2-2.5x increase in performance over Sprites when alpha masking them, and a ~0.5x decrease when alpha blending. However, alpha masking is definitely the more common choice for Sprites, and the alpha blending performance should be improved significantly in the near future as part of Mesh2d improvements tracking issues #13265.

Problems

These are a few notable problems that should ideally be resolved before fully replacing Sprite with the new SpriteMesh:

• The TextureAtlasLayout is an asset that the SpriteMaterial depends on to generate its BindGroup. It is currently just read when the SpriteMesh is added (or changed) and baked into the SpriteMaterial, which means there's no hot reloading for it (changing a TextureAtlasLayout won't update the material). I believe this is best fixed by reading the TextureAtlasLayout asset inside as_bind_group_shader_type(), same as the Sprite's Image.

• Creating a new SpriteMaterial for each new SpriteMesh and its changes is really bad for performance, which is why I've added a rudimentary material caching solution through a HashMap<SpriteMesh, Handle<SpriteMaterial>. However, I do believe there's significant potential for further optimizations in this area.

• Currently, a significant chunk of SpriteMaterialUniform (about 40 bytes) is taken by data used for slicing. This data is useless for Sprites which don't use slicing, and so it adds overhead. Ideally, this data would be placed in a separate binding that is only written and read on the GPU if a certain ShaderDef is set.

[github-actions]

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

[alice-i-cecile]

Can you do some benchmarking to see how performance compares before and after this change? We should have suitable stress tests; a before/after frame time histogram using tracy would be ideal.

[IceSentry]

I believe this should say translucent pixels? Transparent would mean a pixel with an alpha of 0 and in those case alpha masking should be used.

[IceSentry]

I know this should realistically never fail, but I'd prefer if you used an if let and either log an error or do nothing.

[IceSentry]

LGTM This is a very good starting point towards moving Sprites to Mesh2d. Thank you again for working on this!

To give more numbers, on my machine when running bevymark with 120k entities I get 61fps with Sprite and 97fps with SpriteMesh (using AlphaMask)

[tychedelia]

This looks good as a start. I'm a little apprehensive about merging this prior to our 2d->3d rework being complete but also think it could be helpful as a migration target for me once that work gets started in earnest. Since this is relatively easy to back out and it's early enough in the cycle, I'm in favor of merging this now in its temporary state.