Skip to content

Handle different number of classes between y_true and y_pred #37

@valosekj

Description

@valosekj

Apologies for the delay, wanted to really dig through everything before accepting this PR.

I don't think this is going to work long-term, as there is an edge case this does not account for currently. Namely, where their are unique classes in both the training and testing data. For example, if training has classes [A, B, C], and the testing has [A, B, D], right now the result for predict_proba for class C will be used to 'represent' the probability for class D in the testing dataset, which is inherently incorrect.

This is actually an issue with most of the supervised categorical metrics at the moment, and as such we will need to look into a more universal solution here... I'll dig into it later this week, but for now I don't think its smart to push this PR.

Originally posted by @SomeoneInParticular in #12 (comment)

I've moved the discussion about a different number of classes between y_true and y_pred into an issue as the original PR #12 now also contains other unrelated commits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions