llama-index==0.8.54 run Anthropic raise 500 error always

[liujuanjuan1984]

Pls anyone could help,

I run these code, it always raise 500 error, no matter how many times retried.
But it worked about 24h hours ago.

What was happened? How can I fix it?
(I can use my anthropic api-key with the anthropic model directly, it works well.)

from llama_index.llms.anthropic import Anthropic
llm = Anthropic(api_key='my-key')
resp = llm.complete("Test message 1")

Error:

anthropic.InternalServerError: Error code: 500 - {'type': 'error', 'error': {'type': 'api_error', 'message': 'Internal server error'}}

Details:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/llama_index/llms/base.py", line 312, in wrapped_llm_predict
    f_return_val = f(_self, *args, **kwargs)
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/llama_index/llms/anthropic.py", line 140, in complete
    return complete_fn(prompt, **kwargs)
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/llama_index/llms/generic_utils.py", line 153, in wrapper
    chat_response = func(messages, **kwargs)
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/llama_index/llms/base.py", line 186, in wrapped_llm_chat
    f_return_val = f(_self, messages, **kwargs)
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/llama_index/llms/anthropic.py", line 127, in chat
    response = self._client.completions.create(
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/anthropic/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/anthropic/resources/completions.py", line 385, in create
    return self._post(
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 1203, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 892, in request
    return self._request(
  File "/home/ubuntu/my_project_backend/venv/lib/python3.10/site-packages/anthropic/_base_client.py", line 983, in _request
    raise self._make_status_error_from_response(err.response) from None
anthropic.InternalServerError: Error code: 500 - {'type': 'error', 'error': {'type': 'api_error', 'message': 'Internal server error'}}

[dosubot[bot]]

Hey @liujuanjuan1984! 👋 I'm here to help you with any bugs, questions, or even becoming a contributor!

The 500 Internal Server Error you're encountering typically indicates an issue on the server side of the Anthropic API. Here are a few steps you can take to troubleshoot and potentially resolve the issue:

1. Check for Service Outages: Verify if there are any ongoing service outages or issues with the Anthropic API. This can often be found on their status page or through any official communication channels.

2. Retry Logic: Although you mentioned retries, ensure that your retry logic is correctly implemented. The Anthropic class in llama-index has a max_retries parameter, which defaults to 10. You might want to adjust this or add exponential backoff to your retry strategy.

3. API Key and Configuration: Double-check that your API key and any other configuration settings are correct. Since you mentioned the key works directly with the Anthropic model, this might not be the issue, but it's worth verifying.

4. Version Changes: Since the issue started recently, consider any changes in the llama-index or Anthropic API versions. The llama-index==0.8.54 version added support for Anthropic Claude Haiku 3.5, but there are no known issues reported with this version related to Anthropic integration ^[1].

5. Contact Support: If the issue persists and seems to be on the Anthropic side, reaching out to their support team might provide more insights or a resolution.

If none of these steps resolve the issue, it might be a temporary problem with the API that could resolve itself.

To continue talking to Dosu, mention @dosu.