Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable NNPA saturation by default and change the option to --nnpa-disable-saturation #3101

Merged
merged 5 commits into from
Mar 26, 2025

Conversation

tungld
Copy link
Collaborator

@tungld tungld commented Mar 21, 2025

This patch enables NNPA saturation by default to avoid out of range errors in running some models.
The performance penalty is quite small, for example, for roberta base:

Without saturation:

  Statistics 1 (excluding warmup), min, 8.873799e-01, max, 9.775084e-01, mean, 9.113163e-01, stdev, 1.960276e-02
  Statistics 2 (no warmup/quart.), min, 8.993821e-01, max, 9.181984e-01, mean, 9.091503e-01, stdev, 5.989598e-03

With saturation:

  Statistics 1 (excluding warmup), min, 8.923542e-01, max, 9.461253e-01, mean, 9.196692e-01, stdev, 1.480647e-02
  Statistics 2 (no warmup/quart.), min, 9.126368e-01, max, 9.281075e-01, mean, 9.210191e-01, stdev, 5.047690e-03

Copy link
Collaborator

@chentong319 chentong319 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we change the attribute from "saturation" to "nonsaturation"? Will that save some modifications in the test case and code gen?

Copy link
Collaborator

@chentong319 chentong319 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tungld
Copy link
Collaborator Author

tungld commented Mar 25, 2025

What if we change the attribute from "saturation" to "nonsaturation"? Will that save some modifications in the test case and code gen?

Looks a good idea. Let me give it a try. Thanks!

@tungld
Copy link
Collaborator Author

tungld commented Mar 25, 2025

@jenkins-droid test this please

@Sunny-Anand
Copy link
Collaborator

@jenkins-droid test this please

@tungld
Copy link
Collaborator Author

tungld commented Mar 26, 2025

@jenkins-droid test this please

@tungld
Copy link
Collaborator Author

tungld commented Mar 26, 2025

What if we change the attribute from "saturation" to "nonsaturation"? Will that save some modifications in the test case and code gen?

@chentong319 I changed "saturation" to nosaturation". Thanks!

@tungld tungld merged commit 5c6c7a3 into onnx:main Mar 26, 2025
7 checks passed
@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #15390 [push] Enable NNPA saturation b... started at 00:22

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #16409 [push] Enable NNPA saturation b... started at 00:09

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #16407 [push] Enable NNPA saturation b... started at 23:09

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #16407 [push] Enable NNPA saturation b... passed after 1 hr 38 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #16409 [push] Enable NNPA saturation b... passed after 2 hr 2 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #15390 [push] Enable NNPA saturation b... aborted after 2 hr 27 min

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants