Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doubts about the Acceptable Range of Model Accuracy Degradation and Evaluation Metrics in the Paper #214

Open
mia1460 opened this issue Mar 9, 2025 · 1 comment

Comments

@mia1460
Copy link

mia1460 commented Mar 9, 2025

Hello, while reading the paper published by your team, I noticed the content related to "negligible regression" mentioned in Table 3.

Image

  • Question 1: The acceptable range of accuracy degradation is not clearly stated in the paper. I would like to know what specific interval is considered negligible?
  • Question 2: When evaluating the model quality to determine the situation of accuracy degradation, were metrics such as NDCG, HR, and MRR used?

Looking forward to your reply. Thank you!

@jiaqizhai
Copy link
Contributor

Hi - this is discussed in Section 4.2 and Appendix F. For those experiments, we measure model quality using Normalized Entropy (normalized log loss).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants