Increase number raster layers that can be added to a view

[jailln]

Context

Currently, the number of layers we can add to a view is limited for several reasons:

1. This hard limit of 15.
2. The max number of texture units supported by the gpu, computed here (i.e. max number of textures that can be passed to the shader in one rendering pass).
3. The following method that counts the number of textures per tile.

Current GPUs with WebGL2 generally have a max number of texture units >= 32. The method 3 is quite conservative by approximating the number of textures based on the crs and by taking the worst case scenario. In this context, the limit is quickly reached: between 5 to 10 layers from my tests, which is quite limiting for users.

Description of the proposal

The proposal is to overcome these limits, by implementing a method to bind more textures to the layer.

Identified use-cases

Adding more layers to the view.

Implementation

This is only a proposition and should be discussed / specified.

Regarding the methods to pass more textures to the shader, I've found two possibilities:

• Use texture arrays (see this tutorial for instance)
• Create a texture atlas

Texture arrays are available since WebGL 2 and iTowns supports WebGL 2 and dropped WebGL 1 support so it seems to be a good option. However, mind the following issue (that may not be a concern since we have our own implementation of loop unrolling in iTowns).

Regarding the above-mentionned limits:

• Limit 1. : I'm not sure it is still meaningfull. The getColorAtIdUv method is not used anymore and I don't really understand why there would be such a limit of 15. I tried removing it and it works well. I'm curious on having your opinion on this @Desplandis @HoloTheDrunk @gchoqueux
• Limit 2. : seems legit. Could be removed if we use texture arrays though ? If we use texture atlases, we may use it to know when to start creating atlases (when the number of textures added to a tile is greater than this limit).
• Limit 3. : This computation seems quite conservative and we could remove it and compute the real number of texture per material in LayeredMaterial.addColorTile if we use texture atlases. In the case of texture arrays I think we won't need it.

[Desplandis]

Limit 1 comes from the maximum number of texture units supported by WebGL 1.0 (16 I think?). The rationale here is since one is potentially used by the elevation layer, this leaves 15 for all color layers. I think we should drop this hard-coded constant since limit 2 enforces that we do not exceed the gl.MAX_TEXTURE_IMAGE_UNITS limit. As for limit 3, a terrain tile can use up to 3 EPSG:3857 textures so it kinda make sense (shall we do the render in a single pass).

I think that the only "blocking" limitation is 2, and I agree with your proposed methods. However, I would add that we could also use megatextures. We should investigate the pro and cons of each methods. @HoloTheDrunk @gchoqueux

[Desplandis]

I will also open a proposal (was discussed during the hackaton) for reworking the render of terrain tiles into one or two pre-pass (atlas/megatexture lookup and blending of color layers together) and the render of tile. Those two proposals will be closely tied but I think we could start with this one.

[mbredif]

However, I would add that we could also use megatextures. We should investigate the pro and cons of each methods.

I am thrilled by this discussion. Texture2Darrays are a good first step and megatextures are definitely the way to go to push the limits of simultaneous layers and further decouple the geometry (from the DTM) from the appearance (color layers) !
Here is a three.js implementation that I am maintaining for resaerch purposes and presented at the last itowns meeting.
code : https://github.com/mbredif/virtual-texturing
demo : http://mbredif.github.io/virtual-texturing
This lib needs an update to the latest three.js version and a loader for COG support but the basic blocks are here (WMTS, WMS, image pyramids, IIP and IIIF sources). In your use case, you already know the required tiles based from the camera parameters, so you can skip the automatic tile determination (based on the analysis of an offline rendering of tile ids) and only use the visualization part (texture sampling).

[Neptilo]

I finally got a first proof-of-concept to work, using texture arrays: commit
Unfortunately, it looks like the only way to construct a texture array from textures loaded by THREE.TextureLoader is to temporarily draw them on canvases to extract their data. Obviously, this is extremely resource-intensive and makes the application barely usable.
Maybe we could load the data into buffers instead of textures, and construct the texture array without needing to draw canvases, but this would imply a big code overhaul.
Another lead would be to draw the textures on a WebGL gl.framebufferTextureLayer, but Three.js’s render targets don't expose that functionality easily.

[mbredif]

Hello, I had a PR (mrdoob/three.js#22089) on three.js to extend its functionnality with respect to MRT, texture2DArrays and textureCubes that feels similar or at least related, as far as I recall, to the missing APIs you are experiencing.
It was too huge of a PR, but if you want to scavenge some parts, feel free ;)

[Neptilo]

@mbredif Thank you for your suggestion.
For the current issue, a solution was achieved with direct calls to the WebGL API without too much complexity, so I think I will stick to that for now.
Once Three adds official support for texture arrays, then it is worth updating it.