-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Apologies for the delay, wanted to really dig through everything before accepting this PR.
I don't think this is going to work long-term, as there is an edge case this does not account for currently. Namely, where their are unique classes in both the training and testing data. For example, if training has classes [
A
,B
,C
], and the testing has [A
,B
,D
], right now the result forpredict_proba
for classC
will be used to 'represent' the probability for classD
in the testing dataset, which is inherently incorrect.This is actually an issue with most of the supervised categorical metrics at the moment, and as such we will need to look into a more universal solution here... I'll dig into it later this week, but for now I don't think its smart to push this PR.
Originally posted by @SomeoneInParticular in #12 (comment)
I've moved the discussion about a different number of classes between y_true and y_pred into an issue as the original PR #12 now also contains other unrelated commits.