Serious Issue: Frequent and Critical Loss of Chat History/Logs

[m-masahiro-addness]

Type of Documentation Change

Error/Typo fix

Documentation Page URL or Path

No response

Current Content

We are reporting a serious and critical issue regarding the stability and reliability of the Dify platform.

We are observing that the chat history/logs are disappearing with significant frequency. This is not an isolated or rare event; it is happening consistently enough to severely impact our operation and ability to track user interactions and model performance.

Key Concerns:

Data Loss: The disappearance of history represents a critical loss of operational data and diagnostic information.

Operational Instability: The platform's perceived reliability is undermined, making it difficult to trust the system for long-term data storage.

Difficulty in Debugging/Monitoring: Without reliable logs, it is nearly impossible for us to monitor model behavior, debug issues, or conduct necessary audits.

This issue is occurring quite frequently (or "at a quite high frequency") and needs to be addressed with high priority and urgency. We request an immediate investigation into the root cause of this frequent data loss.

Suggested Changes

Suggested Changes / Requested Actions

1. Audit Data Persistence and Retention Policy:
   • Please conduct an urgent audit of the database write operations, data retention settings, and any automated cleanup scripts related to chat history. We need to confirm that chat logs are not being accidentally or prematurely deleted.
2. Enhance Server-Side Logging:
   • Increase the verbosity of server-side logs, specifically for database WRITE and DELETE operations on chat history tables, to help pinpoint the exact moment and reason for the data loss.
3. Implement Immediate Write Confirmation:
   • As a short-term measure, consider implementing a verification step in the backend to ensure a new chat entry is successfully persisted to the database immediately after the response is generated.
4. Collaboration on Diagnostics:
   • We are ready to provide specific timestamps, operational environment details, and relevant log snippets from our side (e.g., front-end network traffic) to assist your engineering team in reproducing and diagnosing the issue.

Reason for Change

Reason for Reporting / Impact of the Issue

This issue must be treated as a high-priority incident due to the following critical impacts:

• Critical Data Loss: The frequent disappearance of chat history results in a critical loss of operational logs and diagnostic data, which are indispensable for debugging, performance monitoring, and model improvement.
• Erosion of Trust: Consistent data loss severely undermines the reliability of the Dify platform, making it unreliable for long-term data tracking and archival purposes.
• Operational Risk: The inability to track and reference past user interactions poses a significant risk to our own customer support, auditing processes, and the personalized user experience we aim to provide.

Code of Conduct

• I agree to follow Dify's documentation contribution guidelines
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

[dosubot]

Hi @m-masahiro-addness! I'm Dosu and I’m helping the dify-docs team.

Your report highlights a critical issue with chat history/log loss in Dify. Based on a deep review of the codebase and related issues, chat history is stored in the database (conversations and messages tables), and deletion is user-driven—there are no automated cleanup scripts or retention policies that would periodically remove chat logs unless manually triggered or misconfigured by an administrator. Immediate database commits are used for write operations, but there is no explicit backend verification step or enhanced logging for chat history creation events—logging is only present for deletion operations, which could make it difficult to pinpoint the cause of silent data loss or failed writes see code see code.

There are no recent fixes or features addressing frequent chat history disappearance, and related issues focus on privacy, retention, and operational bugs rather than persistent data loss see related issue. Some platform-specific bugs (e.g., changing user IDs on iOS/macOS) can make chat logs inaccessible but do not delete the underlying data see related issue.

To address your concerns, I recommend: auditing all database write and delete operations for chat history, reviewing any custom scripts or cron jobs that might affect retention, and increasing server-side logging for both creation and deletion events. Implementing a backend verification step after each chat log write would help confirm persistence and catch silent failures. If you have specific timestamps or logs, sharing them could help correlate frontend events with backend operations and identify root causes.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}