Multi-User Conversation Continuity and Session Management

[devin-ai-integration]

Multi-User Conversation Continuity and Session Management

Problem Statement

AgentOps currently lacks a clean mechanism for managing sessions/traces in multi-user conversational AI applications where:

1. Multiple users interact simultaneously with the same agent system
2. Users return to continue previous conversations across different time periods
3. Distinct conversations need clean separation (especially across different users)
4. Ongoing conversations need to be stitched together when users resume them

Current Limitations

Based on the current AgentOps architecture:

• Sessions are designed as singular workflow executions with unique IDs
• The modern trace-based API supports multiple concurrent traces but doesn't provide conversation continuity mechanisms
• Session inheritance exists for cross-process scenarios but not for temporal conversation continuity
• No built-in user context or conversation threading capabilities

Use Case Details

Scenario: Agent embedded in user-facing chat interface

Requirements:

• Users can start new conversations or resume existing ones
• Multiple users can interact with the system simultaneously
• Each user's conversation history should be tracked separately
• Conversation context should persist across user sessions
• Clear separation between different users' interactions

Proposed Solution Approaches

1. Conversation-Aware Session Management

Add conversation context to session/trace management:

# Proposed API
agentops.start_conversation_trace(
    user_id="user123",
    conversation_id="conv456",  # Optional - auto-generated if new
    conversation_metadata={
        "user_context": {...},
        "conversation_type": "support_chat"
    }
)

# Resume existing conversation
agentops.resume_conversation_trace(
    conversation_id="conv456",
    user_id="user123"
)

2. Hierarchical Trace Organization

Implement conversation-level traces that contain multiple interaction traces:

Conversation Trace (conv456)
├── Interaction Trace 1 (initial user message + agent response)
├── Interaction Trace 2 (follow-up message + agent response)
└── Interaction Trace 3 (resumed conversation after time gap)

3. User Context Management

Add user-scoped session management:

# User-scoped operations
agentops.init_user_context(user_id="user123")
agentops.start_user_trace(trace_name="support_inquiry")
agentops.link_to_conversation(conversation_id="conv456")

4. Conversation Metadata and Linking

Enhance trace metadata to support conversation linking:

• conversation_id: Links related traces together
• user_id: Associates traces with specific users
• conversation_sequence: Orders traces within a conversation
• conversation_context: Preserves conversation state across traces

Implementation Considerations

Backward Compatibility

• New conversation features should be opt-in
• Existing session/trace APIs should continue working unchanged
• Legacy session management should remain functional

Performance

• Conversation linking should not impact trace performance
• User context should be efficiently retrievable
• Conversation history queries should be optimized

Data Model

• Conversation metadata storage strategy
• Relationship modeling between users, conversations, and traces
• Conversation state persistence across time gaps

Success Criteria

1. Clean Separation: Different users' conversations are completely isolated
2. Conversation Continuity: Users can resume conversations seamlessly across sessions
3. Concurrent Support: Multiple users can interact simultaneously without interference
4. Context Preservation: Conversation context and history are maintained across time gaps
5. Developer Experience: Simple, intuitive API for conversation management

Related Documentation

• Current Session Management
• Multiple Sessions Usage
• Trace Context Management

Priority

High - This addresses a fundamental limitation for conversational AI applications, which are a major use case for AgentOps.

Labels

• enhancement
• session-management
• conversation-ai
• multi-user
• api-design