Groq as LLM provider

[moneywaters]

Please add Groq as it has fast chips that can run Kimi K2 model fast enough to be comparable with Claude Sonnet being 80% of its price. PLEASE ! I tried adding it via open-ai provider since it uses same api format, but when asked what llm model are you zed replies to me:

I don't actually have direct access to information about what model or version I'm running as. The system that hosts me will select the appropriate OpenAI model (likely GPT-4-turbo or GPT-3.5-turbo) based on availability and the context of our conversation.

So while I can't tell you the exact model version I'm currently running under, you can effectively treat me as a GPT-class large language model with the training cutoff in mid-2023.

[sirkitree]

I was able to hook it up, but tool calling isn't working...

settings:

"language_models": {
    "openai_compatible": {
      "groq": {
        "api_url": "https://api.groq.com/openai/v1",
        "available_models": [
          {
            "name": "moonshotai/kimi-k2-instruct-0905",
            "display_name": "kimi-k2",
            "max_tokens": 262144,
            "max_output_tokens": 16384,
            "max_completion_tokens": 16384,
            "capabilities": {
              "tools": true,
              "images": false,
              "parallel_tool_calls": false,
              "prompt_cache_key": false
            }
          }
        ]
      }
    }
  },
  "agent": {
    "default_model": {
      "provider": "groq",
      "model": "moonshotai/kimi-k2-instruct-0905"
    }
  },

[LivioGama]

Which is strange since it's working through OpenRouter going to Groq...

EDIT: some of them work for me