-
Notifications
You must be signed in to change notification settings - Fork 289
Open
Description
Hi,
I have installed alpaca_eval-0.6.6 patsy-1.0.1 using pip install git+https://github.com/tatsu-lab/alpaca_eval.
However, when I ran the following code, I got nan for length_controlled_winrate.
ANNOTATORS_CONFIG = "weighted_alpaca_eval_gpt4_turbo"
df_leaderboard, annotations = evaluate( # type: ignore
model_outputs=response_df,
annotators_config=ANNOTATORS_CONFIG,
is_return_instead_of_print=True,
output_path="pneai_base",
).
What is the correct way to get length_controlled_winrate?
Thanks.
Metadata
Metadata
Assignees
Labels
No labels