Implement gradual falloff and blending for light probes. #22610
+628
−169
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, if a fragment overlaps multiple reflection probes and/or irradiance volumes, Bevy arbitrarily chooses one to provide diffuse and/or specular light. This is unsightly. The standard approach is to accumulate radiance and irradiance as a weighted sum. In most engines, light probes have an artist-controllable *falloff* range, which causes the weight of each probe to diminish gradually from the center of the probe. This PR implements both falloff and blending for light probes. Reflection probes and irradiance volumes are blended using a weighted sum. In the case of reflection probes, if the weights sum to less than 1.0, and an environment map is present on the camera, than the environment map receives the remaining weight necessary to bring the total weight up to 1.0. This is useful for reflection probes that correspond to building interiors, to allow smooth transitions between the indoor building and an exterior environment map when exiting the building. Falloff is specified as a fraction of the *interior* of each light probe that applies gradual falloff, instead of specifying a distance *outside* the light probe over which the influence diminishes. (See the documentation comments in `LightProbe` for more detail.) The reason why I chose to do it this way is that the voxel contents of an irradiance volume would be ill-defined within the falloff range otherwise. Clamping to the edge of the 3D voxel cube inside the falloff region (i.e. extending the edge voxels out) is likely to be incorrect, and extending the voxel region to encompass the falloff range plus the interior range would complicate the calculations in the performance-critical PBR shader. TODO: talk about the new example.
alice-i-cecile
added
A-Rendering
refactor the clustering code. At the moment, clustering is a three-step process: 1. `assign_objects_to_clusters` runs on all clusterable objects during the `PostUpdate` schedule, creating lists of all clusterable objects in each view. 2. During the extraction phase, `extract_clusters` runs on all views that had clusters created for them, linearizing the clusters into a list of `ExtractedClusterableObjectElement::ClusterHeader` commands followed by other `ExtractedClusterableObjectElement`s, one for each object in the cluster. Each `ExtractedClusterableObjectElement` specifies the render world entity for the clusterable object. 3. In the render world, `prepare_clusters` processes all `ExtractedClusterableObjectElement` commands to create the GPU buffers, looking up each clustered object in the `GlobalClusterableObjectMeta` table in order to translate from entity to index. Unfortunately, there are two main problems with this: a. Light probes don't have render world entities at all and are instead tracked in `RenderViewLightProbes` components in the render world. Thus step (2) silently fails for them. b. The `GlobalClusterableObjectMeta` table only contains clustered lights, so even if light probes had render-world entities, step (3) would still fail. The end result is that the GPU ends up consulting bogus out-of-bounds indices that may or may not actually refer to the light probe when traversing clusters. This PR fixes the issues: * I extended `extract_clusters` to support light probes, by adding `ExtractedClusterableObjectElement::ReflectionProbe` and `ExtractedClusterableObjectElement::IrradianceVolume` variants. These variants reference the *main* world entities for light probes, since no render-world entities exist for them. * When processing the new `ExtractedClusterableObjectElement` commands, `prepare_clusters` uses the `RenderViewLightProbes` to find the index in the reflection probe or irradiance volume table as appropriate and supply it to the GPU. Note that this step might fail if a texture that the light probe needs hasn't been loaded yet. In this case, an index of -1 is stored, and the shader skips it. This isn't the optimum behavior; ideally we wouldn't cluster such objects at all. However, it was a minimally-invasive change. * I renamed types that referenced clusterable objects to refer to clusterable *lights* specifically if the types only dealt with lights, to reduce confusion in the future. * The `VisibleClusterableObjects` type is currently overloaded to both serve as a component, in which case it contains *all* clusterable objects associated with a view, and to serve as a container for the objects associated with a cluster. Not only is this confusing, but it's also wasteful, as there's bookkeeping that the type does that's not needed when it's serving as a component. I split the type into the component `VisibleClusterableObjects` and the helper structure `ObjectsInCluster`, and encapsulated logic within each type in order to make `assign_objects_to_clusters` easier to understand. * The `gather_light_probes` system performed its own frustum culling on light probes separately from the frustum culling that `assign_objects_to_clusters` also does. This was wasteful and confusing, especially since the frustum culling algorithms differed between the two systems, so I simplified the logic so that `assign_objects_to_clusters` fills out a table for `gather_light_probes` to use. * `compute_radiances` in `environment_map.wgsl` was broken, as it neglected to set `light_from_world` to the identity matrix when falling back to the view environment map when a reflection probe wasn't found. This would cause specular to vanish in some cases (e.g. in the `reflection_probes` example). I fixed the problem. This commit is a prerequisite for bevyengine#22610, as multiple light probes are too broken without it.
Currently, if a fragment overlaps multiple reflection probes and/or irradiance volumes, Bevy arbitrarily chooses one to provide diffuse and/or specular light. This is unsightly. The standard approach is to accumulate radiance and irradiance as a weighted sum. In most engines, light probes have an artist-controllable *falloff* range, which causes the weight of each probe to diminish gradually from the center of the probe. This PR implements both falloff and blending for light probes. Reflection probes and irradiance volumes are blended using a weighted sum. In the case of reflection probes, if the weights sum to less than 1.0, and an environment map is present on the camera, than the environment map receives the remaining weight necessary to bring the total weight up to 1.0. This is useful for reflection probes that correspond to building interiors, to allow smooth transitions between the indoor building and an exterior environment map when exiting the building. Falloff is specified as a fraction of the *interior* of each light probe that applies gradual falloff, instead of specifying a distance *outside* the light probe over which the influence diminishes. (See the documentation comments in `LightProbe` for more detail.) The reason why I chose to do it this way is that the voxel contents of an irradiance volume would be ill-defined within the falloff range otherwise. Clamping to the edge of the 3D voxel cube inside the falloff region (i.e. extending the edge voxels out) is likely to be incorrect, and extending the voxel region to encompass the falloff range plus the interior range would complicate the calculations in the performance-critical PBR shader. TODO: talk about the new example.
pcwalton
added a commit
to pcwalton/bevy
that referenced
this pull request
Jan 21, 2026
refactor the clustering code. At the moment, clustering is a three-step process: 1. `assign_objects_to_clusters` runs on all clusterable objects during the `PostUpdate` schedule, creating lists of all clusterable objects in each view. 2. During the extraction phase, `extract_clusters` runs on all views that had clusters created for them, linearizing the clusters into a list of `ExtractedClusterableObjectElement::ClusterHeader` commands followed by other `ExtractedClusterableObjectElement`s, one for each object in the cluster. Each `ExtractedClusterableObjectElement` specifies the render world entity for the clusterable object. 3. In the render world, `prepare_clusters` processes all `ExtractedClusterableObjectElement` commands to create the GPU buffers, looking up each clustered object in the `GlobalClusterableObjectMeta` table in order to translate from entity to index. Unfortunately, there are two main problems with this: a. Light probes don't have render world entities at all and are instead tracked in `RenderViewLightProbes` components in the render world. Thus step (2) silently fails for them. b. The `GlobalClusterableObjectMeta` table only contains clustered lights, so even if light probes had render-world entities, step (3) would still fail. The end result is that the GPU ends up consulting bogus out-of-bounds indices that may or may not actually refer to the light probe when traversing clusters. This PR fixes the issues: * I extended `extract_clusters` to support light probes, by adding `ExtractedClusterableObjectElement::ReflectionProbe` and `ExtractedClusterableObjectElement::IrradianceVolume` variants. These variants reference the *main* world entities for light probes, since no render-world entities exist for them. * When processing the new `ExtractedClusterableObjectElement` commands, `prepare_clusters` uses the `RenderViewLightProbes` to find the index in the reflection probe or irradiance volume table as appropriate and supply it to the GPU. Note that this step might fail if a texture that the light probe needs hasn't been loaded yet. In this case, an index of -1 is stored, and the shader skips it. This isn't the optimum behavior; ideally we wouldn't cluster such objects at all. However, it was a minimally-invasive change. * I renamed types that referenced clusterable objects to refer to clusterable *lights* specifically if the types only dealt with lights, to reduce confusion in the future. * The `VisibleClusterableObjects` type is currently overloaded to both serve as a component, in which case it contains *all* clusterable objects associated with a view, and to serve as a container for the objects associated with a cluster. Not only is this confusing, but it's also wasteful, as there's bookkeeping that the type does that's not needed when it's serving as a component. I split the type into the component `VisibleClusterableObjects` and the helper structure `ObjectsInCluster`, and encapsulated logic within each type in order to make `assign_objects_to_clusters` easier to understand. * The `gather_light_probes` system performed its own frustum culling on light probes separately from the frustum culling that `assign_objects_to_clusters` also does. This was wasteful and confusing, especially since the frustum culling algorithms differed between the two systems, so I simplified the logic so that `assign_objects_to_clusters` fills out a table for `gather_light_probes` to use. * `compute_radiances` in `environment_map.wgsl` was broken, as it neglected to set `light_from_world` to the identity matrix when falling back to the view environment map when a reflection probe wasn't found. This would cause specular to vanish in some cases (e.g. in the `reflection_probes` example). I fixed the problem. This commit is a prerequisite for bevyengine#22610, as multiple light probes are too broken without it.
pcwalton
added a commit
to pcwalton/bevy
that referenced
this pull request
Jan 21, 2026
refactor the clustering code. At the moment, clustering is a three-step process: 1. `assign_objects_to_clusters` runs on all clusterable objects during the `PostUpdate` schedule, creating lists of all clusterable objects in each view. 2. During the extraction phase, `extract_clusters` runs on all views that had clusters created for them, linearizing the clusters into a list of `ExtractedClusterableObjectElement::ClusterHeader` commands followed by other `ExtractedClusterableObjectElement`s, one for each object in the cluster. Each `ExtractedClusterableObjectElement` specifies the render world entity for the clusterable object. 3. In the render world, `prepare_clusters` processes all `ExtractedClusterableObjectElement` commands to create the GPU buffers, looking up each clustered object in the `GlobalClusterableObjectMeta` table in order to translate from entity to index. Unfortunately, there are two main problems with this: a. Light probes don't have render world entities at all and are instead tracked in `RenderViewLightProbes` components in the render world. Thus step (2) silently fails for them. b. The `GlobalClusterableObjectMeta` table only contains clustered lights, so even if light probes had render-world entities, step (3) would still fail. The end result is that the GPU ends up consulting bogus out-of-bounds indices that may or may not actually refer to the light probe when traversing clusters. This PR fixes the issues: * I extended `extract_clusters` to support light probes, by adding `ExtractedClusterableObjectElement::ReflectionProbe` and `ExtractedClusterableObjectElement::IrradianceVolume` variants. These variants reference the *main* world entities for light probes, since no render-world entities exist for them. * When processing the new `ExtractedClusterableObjectElement` commands, `prepare_clusters` uses the `RenderViewLightProbes` to find the index in the reflection probe or irradiance volume table as appropriate and supply it to the GPU. Note that this step might fail if a texture that the light probe needs hasn't been loaded yet. In this case, an index of -1 is stored, and the shader skips it. This isn't the optimum behavior; ideally we wouldn't cluster such objects at all. However, it was a minimally-invasive change. * I renamed types that referenced clusterable objects to refer to clusterable *lights* specifically if the types only dealt with lights, to reduce confusion in the future. * The `VisibleClusterableObjects` type is currently overloaded to both serve as a component, in which case it contains *all* clusterable objects associated with a view, and to serve as a container for the objects associated with a cluster. Not only is this confusing, but it's also wasteful, as there's bookkeeping that the type does that's not needed when it's serving as a component. I split the type into the component `VisibleClusterableObjects` and the helper structure `ObjectsInCluster`, and encapsulated logic within each type in order to make `assign_objects_to_clusters` easier to understand. * The `gather_light_probes` system performed its own frustum culling on light probes separately from the frustum culling that `assign_objects_to_clusters` also does. This was wasteful and confusing, especially since the frustum culling algorithms differed between the two systems, so I simplified the logic so that `assign_objects_to_clusters` fills out a table for `gather_light_probes` to use. * `compute_radiances` in `environment_map.wgsl` was broken, as it neglected to set `light_from_world` to the identity matrix when falling back to the view environment map when a reflection probe wasn't found. This would cause specular to vanish in some cases (e.g. in the `reflection_probes` example). I fixed the problem. This commit is a prerequisite for bevyengine#22610, as multiple light probes are too broken without it.
pcwalton
added
the
S-Blocked
Contributor
Author
|
Blocked on #22621 |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 22, 2026
…actor the clustering code. (#22621) At the moment, clustering is a three-step process: 1. `assign_objects_to_clusters` runs on all clusterable objects during the `PostUpdate` schedule, creating lists of all clusterable objects in each view. 2. During the extraction phase, `extract_clusters` runs on all views that had clusters created for them, linearizing the clusters into a list of `ExtractedClusterableObjectElement::ClusterHeader` commands followed by other `ExtractedClusterableObjectElement`s, one for each object in the cluster. Each `ExtractedClusterableObjectElement` specifies the render world entity for the clusterable object. 3. In the render world, `prepare_clusters` processes all `ExtractedClusterableObjectElement` commands to create the GPU buffers, looking up each clustered object in the `GlobalClusterableObjectMeta` table in order to translate from entity to index. Unfortunately, there are two main problems with this: a. Light probes don't have render world entities at all and are instead tracked in `RenderViewLightProbes` components in the render world. Thus step (2) silently fails for them. b. The `GlobalClusterableObjectMeta` table only contains clustered lights and decals, so even if light probes had render-world entities, step (3) would still fail. The end result is that the GPU ends up consulting bogus out-of-bounds indices that may or may not actually refer to the light probe when traversing clusters. This PR fixes the issues: * I extended `extract_clusters` to support light probes, by adding `ExtractedClusterableObjectElement::ReflectionProbe` and `ExtractedClusterableObjectElement::IrradianceVolume` variants. These variants reference the *main* world entities for light probes, since no render-world entities exist for them. * When processing the new `ExtractedClusterableObjectElement` commands, `prepare_clusters` uses the `RenderViewLightProbes` to find the index in the reflection probe or irradiance volume table as appropriate and supply it to the GPU. Note that this step might fail if a texture that the light probe needs hasn't been loaded yet. In this case, an index of -1 is stored, and the shader skips it. This isn't the optimum behavior; ideally we wouldn't cluster such objects at all. However, it was a minimally-invasive change. * I renamed types that referenced clusterable objects to refer to clusterable *lights* specifically if the types only dealt with lights, to reduce confusion in the future. * The `VisibleClusterableObjects` type is currently overloaded to both serve as a component, in which case it contains *all* clusterable objects associated with a view, and to serve as a container for the objects associated with a cluster. Not only is this confusing, but it's also wasteful, as there's bookkeeping that the type does that's not needed when it's serving as a component. I split the type into the component `VisibleClusterableObjects` and the helper structure `ObjectsInCluster`, and encapsulated logic within each type in order to make `assign_objects_to_clusters` easier to understand. * The `gather_light_probes` system performed its own frustum culling on light probes separately from the frustum culling that `assign_objects_to_clusters` also does. This was wasteful and confusing, especially since the frustum culling algorithms differed between the two systems, so I simplified the logic so that `assign_objects_to_clusters` fills out a table for `gather_light_probes` to use. * `compute_radiances` in `environment_map.wgsl` was broken, as it neglected to set `light_from_world` to the identity matrix when falling back to the view environment map when a reflection probe wasn't found. This would cause specular to vanish in some cases (e.g. in the `reflection_probes` example). I fixed the problem. This commit is a prerequisite for #22610, as multiple light probes are too broken without it.
pcwalton
removed
the
S-Blocked
Contributor
|
The generated |
pcwalton
added
S-Needs-Review
Contributor
Author
|
This should be ready now. The screenshot shows the sphere partway between the two rooms, with each room's light probe properly blended. Thanks to @atlv24 for the example assets! |
Contributor
Author
|
Not sure what the failure means. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, if a fragment overlaps multiple reflection probes and/or irradiance volumes, Bevy arbitrarily chooses one to provide diffuse and/or specular light. This is unsightly. The standard approach is to accumulate radiance and irradiance as a weighted sum. In most engines, light probes have an artist-controllable falloff range, which causes the weight of each probe to diminish gradually from the center of the probe.
This PR implements both falloff and blending for light probes. Reflection probes and irradiance volumes are blended using a weighted sum. In the case of reflection probes, if the weights sum to less than 1.0, and an environment map is present on the camera, than the environment map receives the remaining weight necessary to bring the total weight up to 1.0. This is useful for reflection probes that correspond to building interiors, to allow smooth transitions between the indoor building and an exterior environment map when exiting the building.
Falloff is specified as a fraction of the interior of each light probe that applies gradual falloff, instead of specifying a distance outside the light probe over which the influence diminishes. (See the documentation comments in
LightProbefor more detail.) The reason why I chose to do it this way is that the voxel contents of an irradiance volume would be ill-defined within the falloff range otherwise. Clamping to the edge of the 3D voxel cube inside the falloff region (i.e. extending the edge voxels out) is likely to be incorrect, and extending the voxel region to encompass the falloff range plus the interior range would complicate the calculations in the performance-critical PBR shader.A new example,
light_probe_blending, has been added. This example shows a reflective sphere that moves between two rooms, each of which has a reflection probe with a falloff range, so the sphere smoothly blends between the two. The user can pan and zoom the camera.