Description
Related to discussion in a Stan discourse post
ROS book and some Stan discourse answers show how to make Jacobian adjustment for elpd_loo values if different models use different transformations of the continuous target. I had tested that a simple approach of adjusting the pointwise values as
loo_2_with_jacobian <- loo_2
loo_2_with_jacobian$pointwise[,1] <- loo_2_with_jacobian$pointwise[,1] - log(2*sqrt(data$y))
works when comparing two models. In case of more than two models, the summary values in $estimates slot are used to sort the models, which causes problems if the order is different after Jacobian adjustment. A quick fix is to update the summary, too
loo_2_with_jacobian$estimates["elpd_loo",] <- loo:::table_of_estimates(loo_2_with_jacobian$pointwise[,"elpd_loo", drop=FALSE])
It would be better to make a function that would properly update all the slots, or probably even better to add jacobian argument to loo() function.
This update would not be big, doesn't cause backward compatibility issues, and would make the life of users easier