Anthropic API Overloaded leads to crash

[ErikBjare]

Traceback (most recent call last):
  File "/home/erb/.local/bin/gptme", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fast/erb/Programming/gptme/gptme/cli.py", line 268, in main
    chat(
  File "/fast/erb/Programming/gptme/gptme/chat.py", line 179, in chat
    for msg in step(
  File "/fast/erb/Programming/gptme/gptme/chat.py", line 230, in step
    msg_response = reply(msgs, get_model().model, stream, tools)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fast/erb/Programming/gptme/gptme/llm/__init__.py", line 57, in reply
    return _reply_stream(messages, model, tools)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fast/erb/Programming/gptme/gptme/llm/__init__.py", line 102, in _reply_stream
    for char in (
  File "/fast/erb/Programming/gptme/gptme/llm/__init__.py", line 102, in <genexpr>
    for char in (
                ^
  File "/fast/erb/Programming/gptme/gptme/llm/llm_anthropic.py", line 95, in stream
    for chunk in stream:
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/anthropic/lib/streaming/_prompt_caching_beta_messages.py", line 62, in __iter__
    for item in self._iterator:
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/anthropic/lib/streaming/_prompt_caching_beta_messages.py", line 122, in __stream__
    for sse_event in self._raw_stream:
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/anthropic/_streaming.py", line 68, in __iter__
    for item in self._iterator:
  File "/home/erb/.local/pipx/venvs/gptme/lib/python3.11/site-packages/anthropic/_streaming.py", line 110, in __stream__
    raise self._client._make_status_error(
anthropic.APIStatusError: {'type': 'error', 'error': {'details': None, 'type': 'overloaded_error', 'message': 'Overloaded'}}

Should just catch it and retry.

[ErikBjare]

Fixed in #356