Event-Driven RAG Framework LLMs

[nafshar]

I have been trying to replicate the Event-Driven RAG based workflow system, recently demonstrated by LlamaIndex :

https://learn.deeplearning.ai/courses/event-driven-agentic-document-workflows/lesson/g2gfb/building-a-workflow

I am having some trouble getting the repsonse form either OpenAI llm or Cohere llm. The OpenAI is complaining about "exceeding quota", while only a single request is made, while Cohere appears not to have implemented the astream.complete() method to return a coroutine [not exactly sure about this, but it looks that way].

Using Cohere:
llama_index.core.workflow.errors.WorkflowRuntimeError: Error in step "step_two": object async_generator cannot be used in "await" expression

Here's the code and the line that causes this issue:

async def step_two(self, ctx: Context, ev: FirstEvent) -> SecondEvent:
        llm = Cohere(api_key=CO_API_KEY, model="command-r-plus")
        # Pay attention to this API: astream.complete(). Here "a" is for async.
        generator = await llm.astream_complete(
            "Please give me the first 50 words of Moby Dick, a book in the public domain."
        )

        async for response in generator:
             # Allow the workflow to stream this piece of response
             ctx.write_event_to_stream(TextEvent(delta=response.delta))

        return SecondEvent(
            second_output="Second step complete, full response attached",
            response=str(response),
        )

It appears Cohere may have not implemented the astream.complete() function quite right as it may bot be returning a coroutine.

This same code, when using OpenAI llm complains about exceeding quota, when only a single call is made to the llm, all be it, we do pull the response form the llm in "delta" sections.

[nafshar]

apparently there is an open issue on this subject which as yet has not been resolves (since September 2024). I would be grateful if anyone at Cohere or LlamaIndex could address this issue.

See : #15843