sft-vs-rl

We evaluate the limitations of RL by comparing it to pass@k performance.

Name		Name	Last commit message	Last commit date
Latest commit History 36 Commits
__pycache__		__pycache__
countdown_evaluation_output		countdown_evaluation_output
data		data
results		results
results_new		results_new
.gitignore		.gitignore
README.md		README.md
correct_samples.jsonl		correct_samples.jsonl
countdown.py		countdown.py
countdown_verifier.py		countdown_verifier.py
evaluate_data.py		evaluate_data.py
evaluate_pass_k.py		evaluate_pass_k.py
generate_verify.py		generate_verify.py
plot_pass_k.py		plot_pass_k.py
prepare_finetuning_data.ipynb		prepare_finetuning_data.ipynb
run_evaluation.sh		run_evaluation.sh
submit_multi_gpu.sh		submit_multi_gpu.sh
test.parquet		test.parquet
test_base.parquet		test_base.parquet
train.parquet		train.parquet
train_new.parquet		train_new.parquet
verifier_test.py		verifier_test.py
vllm_sample_cd.py		vllm_sample_cd.py
vllm_with_verifier.py		vllm_with_verifier.py

Provide feedback