Snapchat and Filament

[elisemorysc]

Hi Devs,

We've recently released 3D bitmoji for our users in profile page:
https://www.techradar.com/news/you-can-now-add-3d-bitmoji-avatar-to-your-snapchat-profile-heres-how

The feature is using server side rendering using Filament (with a few customized changes).
It currently renders almost 1b images per day using a few hundreds GPUs in our backend.

We're very excited with the results, and wanted to thank everyone working on this project!

[romainguy]

Thanks for letting us know @elisemorysc, that's fantastic! I'm curious about the customizations you made, is there anything missing in Filament that you believe we should add?

[elisemorysc]

We've added a few things:

• Support for EGL on Linux, since we work on headless server side (see EGL for OpenGL Linux support #3912). We also wanted to support transparent output since we compose the backgrounds client side.
• The ability to add animations to existing assets. This allows us to load the model once and render multiple poses (see the link above)
• Support CPU morph targets to extend the GPU limit.
• Support for hunter package management to have better control on 3rd party packages. This is public on hunter.

For future versions we plan to support GPU decoding of textures to speed texture loading and to move textures decoding out of the render loop.

[pixelflinger]

Thank you for letting us know, it's very appreciated :-)

It seems like supporting EGL on linux is something we can/should handle (actually I thought we did!). Would you contribute those changes? Is there anything specific to your systems?

[romainguy]

Thanks for the list. We could definitely discuss upstreaming some of those changes if you are interested. Lifting the morph target limit is something we'd like to address, it just hasn't been a high priority so far. As for GPU decoding of textures are you talking about decoding JPEG/PNG/etc. or are you talking about compressed textures? I want to add support for Basis eventually, would that be a good fit for you?

[elisemorysc]

We can definitely submit an EGL (and others changes) PRs. There is nothing specific to our system, although I've never tested it on non nvidia GPUs.
It's based on this approach:
https://developer.nvidia.com/blog/egl-eye-opengl-visualization-without-x-server/

Regarding GPU decoding, I'm talking about JPEG/PNG decoding.
Our system has two parts which need to happen every time the user create/update their avatar:

• Generate the gltf with all the textures. (done using a propriety engine we've built)
• Render the gltf with all the different poses, using filament. There are currently 12 poses (check the app to see them)
  Native GPU textures formats can save time when rendering the gltf. However the formats are expensive the encode and currently there are no GPU encoders. It's a good option when you need to generate once and render many, but not for us. Since NVIDIA has both JPEG GPU encoder and decoders we prefer to use these instead to speed up the process

[rbsheth]

We're actually testing basis_universal through KTX2 right now, so KTX2 support would probably be something we're interested in as well.