Pxc 4469 sst clone #1961

Tusamarco · 2024-10-07T17:08:10Z

Adding preview POC for the Clone method for SST.
The script is based on the one from codership but then need to refactor almost in full to have it working with PXC.
birdeye flow

#              ┌──────────────────┐                      
#              │  Joiner starts   │                      
#              └────────┬─────────┘                      
#              ┌────────▼─────────┐                      
#              │open NetCat listnr│                      
#              └────────┬─────────┘                      
#              ┌────────▼─────────┐                      
#              │Send message to Dn│                      
#              └────────┬─────────┘                      
#              ┌────────▼─────────┐                      
#              │Donor decide IST  │                      
#              │or SST. Send msg  │                      
#              │through NC and    │                      
#              │waits for Joiner  │                      
#              │Clone instance    │                      
#              └────────┬─────────┘                      
#              ┌────────▼─────────┐                      
#              │Joiner get IST    │                      
#              │or SST.If IST     │                      
#              │Bypass all and    │                      
#              │Wait for IST.     │                      
#              │SST Start clone   │                      
#              │Instance and wait │                      
#              │for donor to clone│                      
#              └─────────┬────────┘                      
#              ┌─────────▼─────────────┐                 
#              │Clone process is       │                 
#              │reported.              │                 
#              │When done Dn waits     │                 
#              │Joiner close instance  │                 
#              │And performs 3 restarts│                 
#              └──────────┬────────────┘                 
#              ┌──────────▼──────────────┐               
#              │1) To fix dictionary     │               
#              │2) To recover position   │               
#              │3) Final cleanup         │               
#              └──────────┬──────────────┘               
#              ┌──────────▼──────────────┐               
#              │Send final ready signal  │               
#              │Waits for IST            │               
#              └─────────────────────────┘

This version has a patch to go straight to ist so is not fully functional unless removing that bypass

SSL is NOT supported

bug. Wrong comma

fixing config missed wsrep allowed methods

…tcat is not running

it-percona-cla · 2024-10-07T17:08:15Z

All committers have signed the CLA.

Tusamarco · 2024-10-07T17:18:36Z

I have a question...
the two file wsrep_clone and wsrep_common_clone are not copied over when I do make install where is that defined?
@kamil-holubicki @venkatesh-prasad-v

Need to ad extension or make script will not identify them correctly

…the user Some edit/cleanup as suggested by Kamil

kamil-holubicki · 2024-11-12T17:00:37Z

mysql-test/suite/galera/include/galera_st_kill_slave_ddl.inc

@@ -1,7 +1,7 @@
 --echo Performing State Transfer on a server that has been killed and restarted
 --echo while a DDL was in progress on it

--source include/have_debug.inc
+#source include/have_debug.inc


I think it will cause another tests to fail

kamil-holubicki · 2024-11-12T17:04:02Z

mysql-test/suite/galera/t/galera_sst_clone.cnf

+[mysqld.2]
+wsrep_provider_options='[email protected].#galera_port;gcache.size=1;pc.ignore_sb=true'
+
+[sst]


MTR framework allows running tests in parallel. It allocates groups of ports dynamically for MTR workers.
We can't hardcode ports here. It will prevent MTR framework to run tests in parallel, like
./mtr galera_sst_clone{,,,,,} --parallel=2
Please check how 'mysqld.1.#sst_port' is handled for example

Where @kamil-holubicki do you have a link to file or something?

kamil-holubicki · 2024-11-12T17:05:01Z

mysql-test/suite/galera/t/galera_sst_clone.cnf

+[sst]
+netcat_port=4442
+wsrep-debug=true
+clone_instance_port=4444


I'm not sure we need clone_instance_port parameter at all. Please see following comments in sst script for justification/reasoning.

we do.
This may go through firewall in real life, and the use of port and ranges can be regulated. So better to keep the flexibility to define ports from the beginning.

We already have a way to configure this port.
See my other comment about wsrep_sst_receive_address

mysql-test/suite/galera/t/galera_sst_clone.test

kamil-holubicki · 2024-11-12T17:08:02Z

scripts/wsrep_sst_common_clone.sh

+        WSREP_SST_OPT_PSWD="$2"
+        shift
+        ;;
+    '--port')


This parameter (and some other) is passed to SST script only in case of mysqldump_sst script. ([LINK](These parameters are passed only to the wsrep_sst_mysqldump state transfer script by both the sending and receiving nodes:)) I think we could remove them from clone-specific file

true, this can be cleanup agreed.

scripts/wsrep_sst_clone.sh

kamil-holubicki · 2024-11-12T19:01:22Z

scripts/mysql_system_users.sql

+
+
+
+GRANT ALTER, CREATE, SELECT, INSERT ON PERCONA_SCHEMA.xtrabackup_history TO 'mysql.pxc.sst.role'@localhost;


Why do we need this?

No Idea it was there, not related to changes I have add.

kamil-holubicki · 2024-11-12T19:02:00Z

sql/wsrep_sst.cc

@@ -1230,6 +1230,8 @@ int wsrep_remove_sst_user(bool initialize_thread) {
                        nullptr,
                        "SET SESSION lock_wait_timeout = 1;",
                        nullptr,
+                        "REVOKE mysql.pxc.sst.role FROM 'mysql.pxc.sst.user'@'localhost';",


Why is this needed?

We discuss this.
The correct way to remove the user when having a role, is to revoke the role and then drop the user.
Just dropping the user will work, but the role cleanup is delegated to MySQL, and as we have identified with mtr tests on very fast machines this looks like to be asynchronous.
So to avoid problems and to be on the safe side, we should operate following the right execution. Role first then drop user.

scripts/mysql_system_users.sql

sql/wsrep_sst.cc

scripts/wsrep_sst_common_clone.sh

kamil-holubicki · 2024-11-12T19:08:36Z

As we spoke today, we need to test it with encrypted tables and keyring component/plugin enabled

Tusamarco and others added 20 commits August 2, 2024 15:26

inserting clone scripts

8f5dfa0

Working version of clone

13e77e4

This version has a patch to go straight to ist so is not fully functional unless removing that bypass

First version that works SST and IST

876f274

Working copy of wsrep_clone with PXC

0c5887c

SSL is NOT supported

adding needed GRANTS

b721c41

adding MTR tests

d44cab3

commenting unneeded debug

93315c9

fixing debug setting

1c2799b

Update mysql_system_users.sql

b92a948

bug. Wrong comma

INFO not DEBUG

3795899

trying to identify why is not reporting the clone status in the mtr test

9c852c8

fixing config missed wsrep allowed methods

trying to identify why is not reporting the clone status in the mtr test

d7bab63

fixing config missed wsrep allowed methods

trying to identify why is not reporting the clone status in the mtr test

f5ff5cc

fixing config missed wsrep allowed methods

Working version also on MTR. I still need to find a way to be sure ne…

e307fc3

…tcat is not running

SST clone with mtr successful

b8ff4e3

pre release

a971965

fix mtr

09d9a78

fixing result

87b3971

clean up and netcat removal

caa7bb1

Adding some help description

c7fdebb

Tusamarco assigned kamil-holubicki Oct 7, 2024

kamil-holubicki requested review from venkatesh-prasad-v and kamil-holubicki October 7, 2024 17:14

Tusamarco and others added 5 commits October 18, 2024 11:39

Change needed by different NetCat implementation on Debian

ab1fa17

Add the clone script to CMake to install them in the right location.

b74a565

Need to ad extension or make script will not identify them correctly

fix in the message

f9cf9d8

adjusting value to 1 given NC RH implementation does not support 0

b28c4d7

Merge branch 'percona:8.0' into PXC-4469-sst-clone

91938a8

Tusamarco and others added 3 commits October 30, 2024 17:51

fix permission adding DELETE

5832eb8

Merge branch 'percona:8.0' into PXC-4469-sst-clone

0f1dae1

Adding revoke on SST user to remove the role association before drop …

db53e22

…the user Some edit/cleanup as suggested by Kamil

kamil-holubicki reviewed Nov 12, 2024

View reviewed changes

scripts/wsrep_sst_common_clone.sh Show resolved Hide resolved

scripts/wsrep_sst_common_clone.sh Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pxc 4469 sst clone #1961

Pxc 4469 sst clone #1961

Tusamarco commented Oct 7, 2024

it-percona-cla commented Oct 7, 2024 •

edited

Loading

Tusamarco commented Oct 7, 2024

kamil-holubicki Nov 12, 2024

kamil-holubicki Nov 12, 2024

Tusamarco Nov 13, 2024

kamil-holubicki Nov 12, 2024

Tusamarco Nov 13, 2024

kamil-holubicki Nov 13, 2024

kamil-holubicki Nov 12, 2024

Tusamarco Nov 13, 2024

kamil-holubicki Nov 12, 2024

Tusamarco Nov 14, 2024

kamil-holubicki Nov 12, 2024

Tusamarco Nov 14, 2024

kamil-holubicki commented Nov 12, 2024




		GRANT ALTER, CREATE, SELECT, INSERT ON PERCONA_SCHEMA.xtrabackup_history TO 'mysql.pxc.sst.role'@localhost;

Pxc 4469 sst clone #1961

Are you sure you want to change the base?

Pxc 4469 sst clone #1961

Conversation

Tusamarco commented Oct 7, 2024

it-percona-cla commented Oct 7, 2024 • edited Loading

Tusamarco commented Oct 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kamil-holubicki commented Nov 12, 2024

it-percona-cla commented Oct 7, 2024 •

edited

Loading