Continuing a Chat by adding new messages.

[aviflombaum]

Imagine a Chat that is already existing and belongs to a user that they would want to continue with (just like in OpenAI interface). You create the Chat once and then you'd want to do something like this:

class ChatJob < ApplicationJob
  queue_as :default

  def perform(user_message_id)
    user_message = UserMessage.find(user_message_id)
    chat = user_message.chat

    chat.ask(user_message.content) do |chunk|
      Turbo::StreamsChannel.broadcast_append_to("chat",
        target: "ai_response_for_user_message_#{user_message.id}",
        partial: "messages/message_chunk",
        locals: {message_chunk: chunk.content})
    end
  end
end

Where a user_message is essentially the next message they sent, but not yet a Message attached to a RubyLLM Chat.

So what you're basically trying to do is load the original RubyLLM Chat and add a message to it based on that user_message content. This next message represents a new response to the chat but because what looks to be happening is because in this process there is only 1 chat.ask, the entire context of the previous chat messages are lost, it's as though this an entirely new response.

Does that make sense? Is this possible?

[aviflombaum]

I'm sorry this works fine.