Skip to content

Azure GPT 4 models need adjusted default_max_tokens #41

@cbagwellfnc

Description

@cbagwellfnc

Describe the bug

gpt-4o, gpt-4o-mini, and gpt-4.1 all have 16K max output tokens but json lists as 20K. Attempting to use results in this error response:

400 Bad Request { "message": "max_tokens is too large: 20000. This model supports at most 16384 completion tokens, whereas you provided 20000.", "type": "invalid_request_error", "param": "max_tokens", "code": "invalid_value" }

Changes made to openai.json in commit 530952c need to be ported to Azure to resolve the issue.

To Reproduce

Steps to reproduce the behavior:

  1. Configure Azure provider
  2. Select gpt-4o model
  3. Send prompt
  4. See error

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions