glTF Interactivity - Behavior Graphs - 3D Tiles extensions for geospatial #809
Description
glTF has explored interactivity for awhile, first considering trigger/action lists and now likely going with behavior graphs (similar to Unreal Engine Blueprints).
Ben Houston gave a talk on this to the Metaverse Standards Forum in June 2023 (video). He developed a small open-source TypeScript library, Behave-Graph, to show how this can work at runtime.
The glTF Behavior Graph can be extended with custom events and actions. It would be interesting to consider what geospatial extensions would be useful for 3D Tiles, e.g., things like fly to, with a use case being Cesium Stories capability (and beyond) using visual scripting. Might be a lot of synergy across visual scripting in Omniverse, Unreal, and Unity.
Dwight Rodgers and Alexey Medvedev gave a talk on glTF interactivity in September 2024 (video). The architecture is quite nice, decoupling the node graph itself from the glTF object model targets. We should consider:
- Extension so that 3D Tiles metadata extensions can be targeted
- Extending the node graph for more complete visual scripting, e.g., dynamic arrays, etc. - as a no-code / low-code open standards approach to building 3D geospatial experiences - basically what Blueprint is to Unreal
Note that the node graph is generic enough that it could even target things like the CesiumJS API.
How would this interact with / complement / replace 3D Tiles declarative styling? #2
- Latest public draft of KHR_interactivity
KHR_interactivitypull request: Add KHR_interactivity draft KhronosGroup/glTF#2293- React based interactivity graph authoring tool - glTF-InteractivityGraph-AuthoringTool