Skip to content

Consider Ordinal handling of training and holdout data with different categories #1598

@tamargrey

Description

@tamargrey

There are situations where we may instantiate an Ordinal logical type for a column based on training data, and then when we see the holdout data, there may be new categories that weren't present in the training data, causing the Ordinal validation to fail.

I'm wondering if it would be worth considering a workaround in the Ordinal validation for this sort of situation. Maybe a handling that just converts new values to be nans or allows the user to specify new values in the order. That said, maybe it should be on the Woodwork user to be aware of this and handling it themself.

This issue may end up coming up in EvalML as we add the OrdinalEncoder and further increase Ordinal usage across EvalML.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions