Skip to content

TASK: Provide parallel tests attempting to replicate deadlocks #5513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: 9.0
Choose a base branch
from

Conversation

mhsdesign
Copy link
Member

In #5510 i attempted to cause a locking problem manually and see how the subscription engine reacts.

Now these tests are actual parallel tests were we thought we could replicate the problem of a ContentStreamWasForked event failing

Failed to insert hierarchy relation: An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

(see slack)

Upgrade instructions

Review instructions

Checklist

  • Code follows the PSR-2 coding style
  • Tests have been created, run and adjusted as needed
  • The PR is created against the lowest maintained branch
  • Reviewer - PR Title is brief but complete and starts with FEATURE|TASK|BUGFIX
  • Reviewer - The first section explains the change briefly for change-logs
  • Reviewer - Breaking Changes are marked with !!! and have upgrade-instructions

Also longer runs under more stress dont cause any exceptions / timeouts or deadlocks

`@@tx_isolation` is `REPEATABLE-READ` my mariadb default
])
));

for ($i = 0; $i <= 500; $i++) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when i increase the number of nodes here, forking just gets really slow but the two processes never timeout.

> An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction
@mhsdesign
Copy link
Member Author

mhsdesign commented Mar 18, 2025

@kitsunet Bernard checked his db production setting and select @@tx_isolation; returns REPEATABLE-READ which is the same i have locally.

With that exact setting and the now added test PublishingDuringPublishingTest which has two monkeys writing to their workspace and directly publishing i can 1 to 1 reproduce

An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

Lets see how we can make that SELECT INSERT query resist this problem.
In case we cant, we should aggressively set the whole dbal transaction to isRollbackOnly to prevent invalid partial state being written.

Edit: Actually its only the same message but not the same point in failure as the bugreport.
See the test fails on my system when trying to update the cs version

$this->updateContentStreamVersion($event->getContentStreamId(), $eventEnvelope->version, $event instanceof PublishableToWorkspaceInterface);
and always DURING command simulation (meaning in the rollback transaction...)
-> if i comment out that logic i get another failure

Failed to add hierarchy relation to database: An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction
also during command simulation ... so it seems two simulation stress tested at once seem baaaad.

Bernhards bug was NOT in a simulation as far as I can think because the subscription had an failure and that can only happen during the actual catchup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant