-
Notifications
You must be signed in to change notification settings - Fork 344
LMKT: Add cultural knowledge remembering and cultural safety application #3736
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Polyguard eclektic benchmark
|
|
||
| return [Stat(MetricName("accuracy")).add(scores["correct"])] | ||
|
|
||
| def evaluate( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has a lot of copy and paste logic from the base evaluate() method. Instead of doing this, you should override evaluate_instances() from EvaluateInstancesMetric instead and put your logic there.
You can look at ClassificationMetric for an example of how to do this.
|
Hi, it's been a month since the last update; are you still working on this? |
Yes, I am still working on this project. There were many other things to do last month, so I haven't had time to revise it. I will restart this week. |
In this PR, I added two more scenarios to LMKT benchmark.
Additionally, I fixed some naming convention for previous scenarios.