Handling Exceptions in forward model evaluation

[ajafarihub]

If the forward (FW) model is nonlinear and the priors are not good enough (they are for example non-informative or include values that are far from target values), it is very likely that the forward simulator diverges with some Exceptions; e.g. Newton-solver did not converge. This would essentially mean that the sampled values of the input parameters do not lead to any likelihood evaluation. In such cases, it is practical to ignore such samples and continue sampling just as normal. This is not currently a feature of Probeye, meaning that any first Exception does throw away all evaluations of FW models, which might have already taken a lot of time.

A possible simple solution would be to consider something like 'allowed_exceptions' as an optional input/attribute for the FW model, which is essentially a list of allowed Exceptions in the FW evaluation. In case any of such Exceptions emerges, proper actions should be done with respect to the current sampled inputs and a normal continuation of the inference.

[joergfunger]

In general, this strorage and reusing of existing runs is quite interesting, but poses many challenges. For the solution you proposed, you can simply do that yourself by just catching the exception within the forward model evaluation. However, I'm not sure what would be the corresponding action, one option would be to return a likelihood of zero thus certainly rejecting the sample in te sampling.

In addition, there is an option to prescribe domains for the parameter that essentially return a zero log prior without evaluation of the likelihood

problem.add_parameter(
name="alpha",
tex=r"$\alpha$",
info="A brief explanation of alpha",
domain="[0, +oo)",
prior=Normal(mean=7.0, std=2.0),
)

[ajafarihub]

Thanks for the useful ideas to overcome this issue. I would say, in most cases the domain feature provides us a simple way to avoid unrealistic samples. On the other hand, I am not sure whether we can still get a diverging FW simulation just due to some specific combination of acceptable parameters all within their specified domains (especially in a nonlinear model).

[joergfunger]

But how would you expect probeye to handle that situation?

[ajafarihub]

I am not sure, also not necessarily saying that probeye should store all simulations that have been done up to a raising error/exception. But perhaps at least it can stop everything and return the results up to that point. Maybe that artificial setting of likelihood to ZERO can be added to probeye with a proper warning feedback. But you are also right, the user can directly do this in his/her FW model implementation. I initially thought , this simple trick can already become a part of probeye, so that the user only provides a list of allowed Exceptions.