You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
arviz.reloo allows for drawing samples from a LOO posterior for all data points for which the PSIS-LOO pareto shape diagnostic indicates that PSIS-LOO gives poor estimates.
We should think carefully about how to support this. In particular, we need to design an API that can be easily extended by e.g. a PPL but is straightforward to use without a PPL. The result should be an AbstractELPDResult, but we need some way to indicate that this elpd result comes from different estimation methods. Hence the LOO posteriors refitted should probably have a missing pareto shape diagnostic.
But more generally, once we have this functionality, running exact LOO, LOGO, or k-fold CV should be possible, so the API should support that as well.
In addition, the functionality in #8 should be supported, but this is more complicated, because posterior predictions may be required, and because the order of operations is reversed. With PSIS-LOO, we compute approximate LOO once and reuse the importance weights for all expectations from LOO posteriors, while with exact LOO, one loops over each data point held out and computes each expectation for each LOO posterior.
The text was updated successfully, but these errors were encountered:
arviz.reloo
allows for drawing samples from a LOO posterior for all data points for which the PSIS-LOO pareto shape diagnostic indicates that PSIS-LOO gives poor estimates.We should think carefully about how to support this. In particular, we need to design an API that can be easily extended by e.g. a PPL but is straightforward to use without a PPL. The result should be an
AbstractELPDResult
, but we need some way to indicate that this elpd result comes from different estimation methods. Hence the LOO posteriors refitted should probably have amissing
pareto shape diagnostic.But more generally, once we have this functionality, running exact LOO, LOGO, or k-fold CV should be possible, so the API should support that as well.
In addition, the functionality in #8 should be supported, but this is more complicated, because posterior predictions may be required, and because the order of operations is reversed. With PSIS-LOO, we compute approximate LOO once and reuse the importance weights for all expectations from LOO posteriors, while with exact LOO, one loops over each data point held out and computes each expectation for each LOO posterior.
The text was updated successfully, but these errors were encountered: