Search results

Add debug mode compatibility and type safety for examples in LiveCodeBenchs

Status: Open.

#136 In mlfoundations/evalchemy;

· dkimds opened on Jul 1, 2025

Add debug mode support for custom benchmarks like AIME24

Status: Open.

#134 In mlfoundations/evalchemy;

· dkimds opened on Jul 1, 2025

per prompt performance and public data

Status: Open.

#132 In mlfoundations/evalchemy;

· chanansh opened on Jun 27, 2025

LCB benchmarks are duplicated

Status: Open.

#129 In mlfoundations/evalchemy;

· slimfrkha opened on Jun 13, 2025

How many responses are sampled per query?

Status: Open.

#105 In mlfoundations/evalchemy;

· noobimp opened on Apr 8, 2025

How to change model's temperature ?

Status: Open.

#99 In mlfoundations/evalchemy;

· hrdkbhatnagar opened on Mar 25, 2025

A suggestion for evalchemy about LiveCodeBench

Status: Open.

#96 In mlfoundations/evalchemy;

· BeastyZ opened on Mar 20, 2025

Log samples to huggingface

Status: Open.

#89 In mlfoundations/evalchemy;

· RyanMarten opened on Mar 5, 2025

How to log model response?

Status: Open.

#84 In mlfoundations/evalchemy;

· TundeAtSN opened on Feb 21, 2025

Support multi-node evaluation with vLLM

Status: Open.

#46 In mlfoundations/evalchemy;

· marianna13 opened on Dec 28, 2024

lm-evaluation-harness dependency conflicts

Status: Open.

#45 In mlfoundations/evalchemy;

· penfever opened on Dec 21, 2024

Race condition when running alpaca eval?

Status: Open.

#31 In mlfoundations/evalchemy;

· GeorgiosSmyrnis opened on Nov 23, 2024