using ScaleAndShiftInvariantLoss in depth_nerfacto

[zhan-xu]

I am trying to apply the ScaleAndShiftInvariantLoss to depth_nerfacto as depth loss. Currently, there is an option that uses depth_ranking_loss with zoe depth. This works fine. I am thinking if I can instead use ScaleAndShiftInvariantLoss with zoe depth here.

What I have done:
In depth_nerfacto, I modified

# loss_dict["depth_ranking"] = np.interp(self.step, [0, 2000], [0, 0.2]) * depth_ranking_loss(
#     outputs["expected_depth"], batch["depth_image"]
# )
depth_gt = batch["depth_image"].to(self.device)[..., None]
depth_pred = outputs["expected_depth"][..., None]
mask = torch.ones_like(depth_gt).reshape(1, 32, -1).bool()
loss_dict["depth_ranking"] = np.interp(self.step, [0, 2000], [0, 0.2]) * self.scale_shift_invariant_loss(
    depth_pred.reshape(1, 32, -1), (depth_gt * 50 + 0.5).reshape(1, 32, -1), mask
)

Here self.scale_shift_invariant_loss = ScaleAndShiftInvariantLoss(alpha=0.5, scales=1)

This doesn't give reasonable results. I specifically have two questions:

1. why the input to "ScaleAndShiftInvariantLoss" has shape "1 32 mult"? What does 32 mean?
2. Do I need to do depth = depth / ray_bundle.metadata["directions_norm"] as done in base_surface_model.py?

Thank you!