Migrate CassandraChatMemory to CassandraChatMemoryRepository

[markpollack]

This would allow us to have a more robust implementation than we currently have based on the ChatMemory.get(conversationId, lastn) interface that is currently deprecated.

This has been done for Jdbc in the PR - as a basis for comparison.

#2890

[linarkou]

As for now, cassandraChatMemory stores data in table with 4 columns like that:

create table ai_chat_memory (
    session_id        text,
    message_timestamp timestamp,
    assistant         text,
    user              text,
    primary key (session_id, message_timestamp)
)
    with clustering order by (message_timestamp desc);

Maybe we should change table structure similar to JdbcChatMemory?

create table ai_chat_memory (
    session_id        text,
    message_timestamp timestamp,
    type              text,
    content           text,
    primary key (session_id, message_timestamp)
)
    with clustering order by (message_timestamp desc);

It'll give us an ability to store tool/system messages, not only user/assistant.

What do you think @markpollack @michaelsembwever ?

[michaelsembwever]

Maybe we should change table structure similar to JdbcChatMemory?

Then the table would need a primary key like

primary key (session_id, message_timestamp, type)

Which can be done, and I have nothing against such an approach.

[linarkou]

Can't session_id+message_timestamp be a primary key? As far as I remember you set unique timestamp.

On other hand, there is #2902 with request on message id field, so we can introduce it here.

[michaelsembwever]

@linarkou , let's create a separate issue for your suggestion, as there's a time-constraint to get this issue fixed.

[michaelsembwever]

@markpollack draft PR @ #3036
eta to finish it tomorrow.

[michaelsembwever]

#3036 is ready for review now.

[markpollack]

fantastic!

[michaelsembwever]

There's a bug here I need to fix. @linarkou statement above

It'll give us an ability to store tool/system messages

is actually mandatory… MessageWindowChatMemory relies on persisting SystemMessages.
working on a fix…

[markpollack]

Thanks @michaelsembwever

[michaelsembwever]

bug (optimisation) fix in progress: #3097
completion eta ~24 hours.

[markpollack]

@michaelsembwever thanks eh!

[markpollack]

@michaelsembwever we can always do a bug fix post RC1 pre GA - just want to make sure we avoid modifying and public facing apis/properties.

[michaelsembwever]

@markpollack , ready for review #3097

[michaelsembwever]

we can always do a bug fix post RC1 pre GA

i don't think before #3097 it would work at all with MessageWindowChatMemory (as it expected to persist SystemMessages).

and just generally, the usage pattern of MessageWindowChatMemory was substantially different: and with Cassandra not being a relationship style db; that it really required a revamp of the schema. #3097 is properly aligned to Cassandra now, provides a number of advantages over other chat-memory implementations (according to its known strengths), and could quite likely outperform latency-wise the other implementations (ofc "it depends").

[markpollack]

Thanks so much for the deep analysis and changes at the last minute.

If you want to advertise this in the docs, it may draw more folks into using cassandra.

merged in 87b680a