ValueError: cannot reindex on an axis with duplicate labels

[Liesy]

I have formatted the outputs like "example/outputs.json" and run the command "alpaca_eval --model_outputs xxx.json", then the value error occurred:

Traceback (most recent call last):
File "/data/liyang/anaconda3/envs/py311torch212/bin/alpaca_eval", line 8, in
sys.exit(main())
^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/main.py", line 611, in main
fire.Fire(evaluate)
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/fire/core.py", line 684, in CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/main.py", line 179, in evaluate
leaderboard[name].update(fn_metric(annotations, **(metric_kwargs or {})))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 95, in get_length_controlled_winrate
df_XY_train, df_X_test, sample_weight = _get_featurized_data(
^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 247, in _get_featurized_data
df_XY_train, df_X_test = make_dmatrix_for_model(df_gamed_and_m, df_test, formula=formula)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/alpaca_eval/metrics/glm_winrate.py", line 281, in make_dmatrix_for_model
df_XY_train[col_y_true] = df_train[col_y_true] # adds the label
~~~~~~~~~~~^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 4311, in setitem
self._set_item(key, value)
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 4524, in _set_item
value, refs = self._sanitize_column(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 5263, in _sanitize_column
return _reindex_for_setitem(value, self.index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 12692, in _reindex_for_setitem
raise err
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/frame.py", line 12687, in _reindex_for_setitem
reindexed_value = value.reindex(index)._values
^^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/series.py", line 5153, in reindex
return super().reindex(
^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/generic.py", line 5610, in reindex
return self._reindex_axes(
^^^^^^^^^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/generic.py", line 5633, in _reindex_axes
new_index, indexer = ax.reindex(
^^^^^^^^^^^
File "/data/liyang/anaconda3/envs/py311torch212/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 4429, in reindex
raise ValueError("cannot reindex on an axis with duplicate labels")
ValueError: cannot reindex on an axis with duplicate labels

and no JSON file was saved.

[Liesy]

My xxx.json:

[
  {
    "instruction": "What are the names of some famous actors that started their careers on Broadway?",
    "output": "test",
    "generator": "test",
    "dataset": "helpful_base"
  }
]