[Question]: Can custom models be used to create the agents?

[mansiibm]

Hi,I have my yaml config set as -

custom:
- name: "Deepseek"
apiKey: "${deepseek_api_key}"
baseURL: "<>"
models:
default: ["DeepSeek-R1"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "DeepSeek-V3-0324"
modelDisplayLabel: "Deepseek"
- name: "xai"
apiKey: "<>"
baseURL: "<>"
models:
default: ["grok-3"]
fetch: false
titleConvo: true
directEndpoint: true
titleModel: "grok-3"
modelDisplayLabel: "xAI"

So ,I have created an env var also as- XAI_API_KEY and passed the key. But when I choose the model grok 3 for agent then it fails with the following message-

So can custom models be used to create agents?

PS- The key is correct as for normal model conversation it works.

[mansiibm]

only these are supported for agents right now, apart from this no other provider or custom even , is not supported hence I was facing the above issue

[danny-avila]

Grok 3 working fine on my end:

Custom endpoint config:

    - name: "xai"
      apiKey: "${XAI_API_KEY}"
      baseURL: "https://api.x.ai/v1"
      models:
        default: ["grok-3-latest"]
        fetch: true
      titleConvo: true
      titleMethod: "completion"
      titleModel: "grok-beta"
      summarize: false
      summaryModel: "grok-beta"
      forcePrompt: false
      modelDisplayLabel: "Grok"

This has nothing to do with allowedProviders and you could specify a custom endpoint by name for that field, the docs are misleading on that point and needs to be corrected.

  agents:
    disableBuilder: false
    allowedProviders: ["openAI", "azureOpenAI", "anthropic", "google", "xai"]

[mansiibm]

Hi Danny thanks for replying instead of base url pointing to Xai instance I am using one provided from azure. I had to set direct endpoint to true for this setting. Instead of beta model my deployment in azure from Xai is ‘grok-3’ named .

[mansiibm]

I am using Xai from azure . In azure I get grok-3 and grok-3-mini. Because of this deployment I have an api key and api base url from azure’s cognitive account ,which I place for base url in above Config and set directEndpoint: true so that LC picks the azure url over the kind of url you used . Please let me know if you need more details . Thanks :)

[mansiibm]

I did mention custom endpoint as xai still did not work with my setting .

[mansiibm]

Hi, it suggests that the configured api key is incorrect and needs to be fetched from xai . But then again as a model in xai endpoint it works absolutely fine

[danny-avila]

Check this thread to see how to configure Grok 3 as provided by Azure:

#7508

[mansiibm]

Hi Danny thankyou for providing the above ticket :) . So, as mentioned there I also created a serverless config in my yaml like this

- group: "serverless-example"
       apiKey: "${xai_api_key}"
       baseURL: "https://<hidden>.api.cognitive.microsoft.com/models/chat/completion"
       version: "2024-05-01-preview"
       serverless: true
       models:
           grok-3: true
           grok-3-mini: true 
           DeepSeek-R1: true

So, using this from azure endpoint. all the models are working with title generation also .
.

Now if I try using with AGENTS , these models- grok-3, grok-3-mini

I am able to chat with it but title is not getting generated.

In the logs its tells me to use- version: " 2024-08-01-preview" this or later but my instance has the version-

https://.api.cognitive.microsoft.com/models/chat/completions?api-version=2024-05-01-preview

and the logs -