Skip to content

Logistic post-transform losing precision on large negative inputs, resulting in TreeEnsembleClassifier reporting low probabilities as 0 #26281

@bentheiii

Description

@bentheiii

Describe the issue

When using TreeEnsembleClassifier with a LOGISTIC post-transform, onnxruntime might report certain probabilities as zero, when the onnx reference evaluation would report them as very small, but non-zero probabilities (example is using python, but recreated with c++ as well)

# with LOGISTIC post-transform
onnxruntime:  array([[1., 0.]], dtype=float32)
onnx reference evaluator:  array([[9.9999994e-01, 3.0348513e-08]], dtype=float32)

If the post-transform is set to NONE, and applying the logistic function manually, the results are closer together

# with NONE post-transform
onnxruntime:  array([[ 17.310535, -17.310535]], dtype=float32)
onnx reference evaluator:  array([[ 18.310518, -17.310518]], dtype=float32)
# applying sigmoid to both results manually produces
onnxruntime:  array([[ 9.99999969652e-01, 3.0348005739e-08]], dtype=float32)
onnx reference evaluator:  array([[ 9.99999988835e-01, -3.034852166e-08]], dtype=float32)
# which are much closer together

I believe that the ORT implementation of the logistic function is less precise than other implementations for large negative inputs.

To reproduce

Although the model I found this cannot be shared, this can be reproduced with any TreeEnsembleClassifier that produces large (non-transformed) probabilities

Urgency

Without this fixed, we now need to disable post-transformation on our models, and applying the logistic post-process manually.

For future reference, the following snippet replaces a logistic post-transform with a sigmoid function

from onnx.helper import make_graph, make_model, make_node, make_tensor_value_info
from onnx import TensorProto, load_model, save_model
from onnx.compose import merge_graphs

def build_with_sigmoid(perm_onnx_path):
    perm_model = load_model(perm_onnx_path)
    perm_model.graph.node[0].attribute[-1].s = b"NONE"
    probs_raw = make_tensor_value_info("probabilities_raw", TensorProto.FLOAT, [1, 2])
    probs_trans = make_tensor_value_info("probabilities_transformed", TensorProto.FLOAT, [1, 2])
    new_graph = make_graph([
        make_node("Sigmoid", ["probabilities_raw"], ["probabilities_transformed"], name="Logistic")
    ], "Logistic", [probs_raw], [probs_trans])
    new_model = make_model(
        merge_graphs(perm_model.graph, new_graph, [("probabilities", "probabilities_raw")]), 
        opset_imports=perm_model.opset_import, ir_version=perm_model.ir_version
    )

    onnx_path = "model_sigmoid.onnx"
    save_model(new_model, onnx_path)

Note that, for reasons I don't have time to get into, the sigmoid is also less precise than running numpy, but it's close enough for my purposes

Platform

Linux

OS Version

Ubuntu 22.04.5

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.23.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions