Unexpected Behavior with clear_foreign_voxels: true After Map Loading

[KuangHaofei]

Hi @Schmluk,

Thanks for your great work!

When I applied your method in my scenario, I enabled clear_foreign_voxels: true due to segmentation errors in some complex scenes. This setting helped remove many outliers from the mesh, as shown in this example (see attached image for details, it generate a clean wall where most of the spurious voxels are cleared):

However, after saving and then loading the .panmap file, the outliers reappeared in the mesh (see the attached image).

Upon reviewing the code, I noticed the following:

1. When clear_foreign_voxels: true, only the mesh generation process removes outliers based on the class layer. Other layers, such as the TSDF layers, remain unchanged. This can be observed in this part of the code:
   Relevant code snippet

2. During a subsequent run, when loading the saved map, the previously initialized class layer seems to be ignored in the mesh generation process, as seen here:
   Relevant code snippet

As a result, the mesh still contains outliers after loading the map.

To address this issue, I have two questions:

1. How can I update the other layers (e.g., TSDF layer or TSDF blocks) when outliers are removed using the clear_foreign_voxels option?
2. How can I ensure the initialized class layer is used during mesh generation after loading the map?

If there are alternative approaches or recommendations, I’d be happy to hear your suggestions.

Thanks in advance for your help!

[Schmluk]

Hi @KuangHaofei ,

I'm not sure if the class layer is being serialized and saved currently. If it is the code snippet you showed should be able to load the layer and apply it as you usually would.

1. You can change how data is integrated into all layers in the projective integrator. Another option (that might already be there, but maybe is not configured) is to apply the classlayer, which has the same effect but bakes the clearing into the TSDF. You could run this prior to saving to get the desired effect.

2. If the class layer is currently not being serialized/you can implement a serialization as in voxblox. Once the class layer is there after loading it should automatically be used as usual.

Hope this helps!