[DocDB] Data inconsistency – Row count drops without workload or deletes on YB 2025.1.1.0-b39

[ShikharSahay]

Jira Link: DB-17524

Description

The error we see is the following for a CDC run -
1.
At 00:46:59 we get -

00:46:22,453:DEBUG: > 172.151.26.239 > /home/yugabyte/tserver/bin/ysqlsh --command "select count(k) from test_cdc_b8bb6f_hash where uuid = 'd541279d-b577-4c27-a970-42085336a7e3'" -h 172.151.28.158,172.151.29.135,172.151.20.59,172.151.17.149,172.151.26.239,172.151.22.82 -d cdc_5414ce -U sql_user_1 -A 2>&1


2025-07-04 00:46:59,835:DEBUG: < 172.151.26.239 < count
188351
(1 row)

At 00:49:34 we get -

00:47:02,836:DEBUG: > 172.151.17.149 > /home/yugabyte/tserver/bin/ysqlsh --command "select count(k) from test_cdc_b8bb6f_hash where uuid = 'd541279d-b577-4c27-a970-42085336a7e3'" -h 172.151.26.239,172.151.28.158,172.151.22.82,172.151.17.149,172.151.20.59,172.151.29.135 -d cdc_5414ce -U sql_user_1 -A 2>&1

00:49:34,648:DEBUG: < 172.151.22.82 < count
188350
(1 row)

Between these two times (00:42:47 - 00:52:09) a rolling restart of VMs 172.151.22.82, 172.151.17.149, 172.151.26.239 occurred.
Here the count abruptly goes from 188351 -> 188350 without any workload/delete operation running.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.