-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Score tasks #2452
base: main
Are you sure you want to change the base?
Score tasks #2452
Conversation
rimashahbazyan
commented
Nov 4, 2024
- Added SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models
- Fixed a bug for generate until tasks to default the "until" parameter to each model's end of sequence.
Hi! looks great mostly! Could you add the following changes:
pip install pre-commit
pre-commit install
pre-commit run --all-files
|
Thanks! |
@rimashahbazyan Thanks! test failing due to a missing function in
|
@baberabb |
postprocess minor fix
postprocess minor fix CR for math added
…-harness into score_tasks
@baberabb I did some small bugfixes, and double-checked everything, this PR is final; I won't commit anything to the branch, when do you think it could be reviewed? |
Sorry for the delay, I'll try merging by next week if thats ok. Wanted to verify no other task is affected by removing the default |