Automation of evaluating new ggml models

[matthoffner]

I'm interested in running lm-eval on ggml models similar to Open LLM Leaderboard: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

I've started a PR to add ggml support to lm-eval (following llama-cpp-python): https://github.com/EleutherAI/lm-evaluation-harness/pull/617/files

Eventually I imagine this could be automated with each @TheBloke release. Any feedback on the PR or help in getting it working would be appreciated!

[ggerganov]

This is probably tangentially related to the plan of adding automated CI in Azure Cloud, so keep an eye on that discussion too:

ggml-org/llama.cpp#1985