Terrain LOD: next-level visual quality and performance improvements

[brendancol]

Overview

The terrain LOD system (rtxpy/viewer/terrain_lod.py, rtxpy/lod.py) is functionally complete with tiling, streaming, hysteresis, frustum culling, and progressive loading. This issue tracks the next round of improvements to push visual quality and performance further.

Tier 1: High Impact, Moderate Effort

3. Terrain-adaptive LOD thresholds

Currently LOD is purely distance-based. A flat salt lake at LOD 0 wastes triangles; a jagged ridgeline at LOD 3 loses critical silhouette detail.

Fix: Compute a per-tile roughness metric (variance of elevation or max deviation from bilinear fit) during pyramid build. Use it to shift LOD thresholds — rough tiles get promoted to finer LOD at greater distance; smooth tiles get demoted earlier. This redistributes the triangle budget where it matters most.

Tier 2: High Impact, Higher Effort

4. Geomorphing (LOD blend transitions)

Even with hysteresis, tiles still pop when they transition. Geomorphing smoothly interpolates vertex Z positions between the old and new LOD over 10-20 frames.

Fix: For each vertex in the finer mesh, store where it would collapse to in the coarser mesh. During transition, lerp Z between the two positions based on a blend factor that ramps 0→1. This eliminates all visible popping. Requires storing both LOD levels' vertex data temporarily and modifying the upload path, but doesn't touch the shader.

5. Cluster GAS for tile batches

Each LOD tile is currently a separate GAS in the IAS. With 50-100+ tiles, IAS traversal overhead grows. The cluster accel code (rtx.py:1120-1179) is already proven for single terrain.

Fix: Batch tiles at the same LOD level into a single cluster GAS to reduce IAS entry count. Requires deferred rebuild that collects same-LOD tiles and rebuilds their cluster GAS when the set changes.

6. Heightfield tiles for LOD 0

LOD 0 tiles are full-resolution regular grids — exactly what add_heightfield_geometry is designed for. Heightfield uses ray marching with bilinear interpolation, giving smooth normals for free and using ~4 bytes/pixel vs ~16 bytes/pixel for explicit triangles.

Fix: Use heightfield for LOD 0 and TIN for LOD 1+. Cuts memory for highest-detail tiles by 4x with better visual quality (bilinear normals). Challenge: heightfield uses a different hit group (__closesthit__heightfield) and different _geom_state fields, so mixing with TIN tiles needs careful SBT indexing.

Tier 3: Polish

7. Edge stitching instead of skirts

Replace skirts with constrained boundary vertices. When adjacent tiles have different LODs, the finer tile's boundary vertices get snapped to the coarser tile's edge positions.

Benefit: Eliminates skirt geometry (~10-15% triangle savings per tile) and removes skirt wall artifacts at grazing angles. The _tile_lods dict already has neighbor LOD info needed at build time.

8. Hybrid LOD for placed geometry

simplify_mesh and build_lod_chain exist in lod.py but aren't used by the tile system. When zarr chunk manager loads meshes (buildings, roads), they could be simplified at load time based on their tile's LOD level.

Fix: LOD 0 chunks get full-detail meshes; LOD 2-3 chunks get 25-50% decimation via simplify_mesh. Keeps overall triangle budget proportional to visual importance.

Suggested priority

1 → 2 → 3 deliver the most visible improvement. Smooth normals alone would transform visual quality. Threaded building eliminates frame hitches during streaming. Adaptive LOD makes the triangle budget smarter. The rest builds on that foundation.