Offset for probalistic modeling - question

[gugatr0n1c]

Hello there,

please, I have question about offset.

I am modeling probability of some event, then I need to run additional model on the top of existing one. Offset is great way how to do it, but I was not able to find how to do it properly and my examples behave really strange.

I am using EvoTreeRegressor, with log_loss loss, target is binary. From first stage model I have probabilities.

Please how do it properly? My current logic is:

a) As offset I am using probabilities (OR should I transform it to logits before?)
b) Should I expect as prediction cummulative value (offset plus new_pred)? Or just new_pred and I need to manually sum offset and new_pred?
c) I would expect cumulative value, but then I need to call predict function with offset as well (for unseen data)? Seems to no way to call predict with offset.
d) Or prediction is in form of logit additional? So need do calc logits from offset, calc sum with new logit pred, and then transform to probabilities?

Actually there is several way how EvoTrees can do it internally. This case with logits is not covered in docs, any info would be great.

Best regards,
Lubomir

[gugatr0n1c]

Ok, I ran several runs to figure it out. And seems to me:

a) using probs (not logits) as offset is correct
b) model predict probabilities but from new addiotional logits only
c) for final prediction I need: sigmoid(logit(offset) + logit(new_pred)) = correct probability
d) it is not very intuitive, but in some sence this makes sence

Can you pls confirm this?

e) It would be better that predict function is forced to has offset input when model was built with offset, otherwise it is quite tricky to know what transformation is needed for which case (logit, log, poisson,...)
f) maybe at least for future users mention this in docs

Best regards,
Lubo

[jeremiedb]

• a: correct, the offset is expected to be in its "natural" basis, that is in the projection that is returned by using predict(model, data).
• b: yes: model with offset can be thought of as a residual/stacking model on top of an an underlying offset.
• c: Yes that seems the correct approach to derive the combined probability.
• d/e: I'm open to suggestions :) Having the offset applied at inference time could make sense, assuming that the offset variable is present in the table on which prediction is called. It however also makes sense to me to have the model treated just as a residual model, hence ignore the "offset" notion at inference time like it's currently the case, and let the user perform the desired transformations. It effectively require the user to understand well the what's happening, and the docs to be explicit about it.
• f: agree that a small tutorial would be a nice addition to illustrate how to use offset, especially with loss function involving reprojection like for logloss

[gugatr0n1c]

Thanx for reply!

Only ad d/e)
Yep, I know understand to logic, then sure - the residual point of view makes somehow sence mainly for linear (MSE) case where one can just sum directly offset and residual (prediction).

But for logloss, probabilities: residual logits (prediction) are transformed to probabilities at inference time and happened somathing like:

• offset: 0.1, 0.15, 0.95 (base init often strong predictions)
• new residual logits: 0.003, -0.002, 0.001 (raw prediction under the hood)
• but then visible predictions at intefrence: 0.50075, 0.4995, 0.50025
• as you can see this predictions of probabilities are quite strange - I cannot sum it, looks crazy compare to offset :)
• extracting valuable part is not very intuitive (but still make sence)

But sure, I agree user should understand what is actually happening, why predictions are strange this way, and what to do next.
I am happy user of EvoTrees, now I understand the logic - in my code, is "one line fix". No big deal. Mayby just improvement in docs is good enough. I actually not strong in internals of EvoTrees to suggest any change for predict functions. Please feel free to close this issue if this is minor.