-
Notifications
You must be signed in to change notification settings - Fork 15
Distributed Transaction
How to ensure high performance, data consistency, and availability in distributed transactions has always been a difficulty (see CAP theory). One of the most basic principles is: If possible, try not to use distributed transaction design. For example, in jSqlBox, try to use TinyTx or GroupTx to manipulate single-machine transactions without introducing distributed transactions. These two transaction managers support sub-database and table division. If the business adopts the Sharding scheme, it can often support large amounts of data. Under the previous topic, it does not create a distributed transaction problem. Of course, this has relatively high requirements for business design and persistence layer tools. All relevant business data needs to be stored in the same database, that is, in the sub-database and table The case will also converge to a single data source for database operations. If the microservices architecture is adopted, all orders are placed in one library, and all users are placed in another library, although the programming is simple, it creates a distributed transaction problem between microservices and has to introduce distributed transactions Tools to solve this problem.
One scheme of distributed transactions is to use XA transactions, which use the database itself to support the XA distributed transaction protocol. In the demo \ jsqlbox-atomikos directory, there is a demonstration of distributed transactions, mainly using the support of Atomikos and Spring for XA transactions. No new things have been invented, just a simple demonstration of the combination of sub-database and sub-table of jSqlBox. Use of XA transactions. XA transactions are based on two-phase commit and need to be supported by the database itself. If the concurrency is large, lock table conflicts may cause performance problems. I do not recommend using XA transactions, not only because of its poor performance, but also its theoretical basis is essentially problematic. For example, the network interruption at the last step of two-phase submission or three-phase submission may still cause data inconsistency.
Seata is an open source distributed transaction tool that uses an automatic compensation scheme. This is an open source project from Alibaba. It is very hot. It is characterized by small business intrusions and automatically creates rollback SQL by analyzing SQL. However, I personally feel that it creates rollback SQL based on analyzing SQL syntax, resulting in poor performance, maintainability, and database compatibility of the entire tool. So far, I feel that it is not fully mature. jSqlBox does not currently support Seata transactions and may be introduced in the future.
Gtx transaction is a distributed transaction module that comes with jSqlBox3.0.0, which makes it possibly the first DAO tool that comes with distributed transactions. Its general idea is similar to Seata, which also automatically rolls back by generating reverse records to reduce business intrusion, but the idea adopted by jSqlBox is to build distributed transactions on top of ORM tools, not to analyze SQL content but Record entity insertion, deletion, and modification operations to generate rollback records. In this way, the difficulty of implementation is reduced by one level to sacrifice the SQL support function to achieve the best database compatibility and support all databases. In terms of specific implementation, it adopts the scheme of maximum guarantee completion mode combined with global record lock. For the architecture, please refer to [Improvement of SAGA Distributed Transactions by Bag Distributed Transactions] 3053815). ! [image] (https://images.gitee.com/uploads/images/2019/0905/171926_90f5f39e_920504.jpeg)
The following is a demonstration of a distributed transaction:
public class GtxTest {
DbContext [] ctx = new DbContext [3];
private static DataSource newTestDataSource () {
HikariDataSource ds = new HikariDataSource ();
ds.setDriverClassName ("org.h2.Driver");
ds.setJdbcUrl ("jdbc: h2: mem:" + new Random (). nextLong () // random h2 ds name
+ "; MODE = MYSQL; DB_CLOSE_DELAY = -1; TRACE_LEVEL_SYSTEM_OUT = 0");
ds.setUsername ("sa");
ds.setPassword ("");
return ds;
}
@Before
public void init () {
DbContextlock = new DbContext (newTestDataSource ());
lock.setName ("lock");
lock.executeDDL (lock.toCreateDDL (GtxId.class));
lock.executeDDL (lock.toCreateDDL (GtxLock.class));
lock.executeDDL (lock.toCreateGtxLogDDL (Usr.class));
GtxConnectionManager lockCM = new GtxConnectionManager (lock);
for (int i = 0; i <3; i ++) {
ctx [i] = new DbContext (newTestDataSource ());
ctx [i] .setName ("db");
ctx [i] .setDbCode (i);
ctx [i] .setConnectionManager (lockCM);
ctx [i] .setMasters (ctx);
ctx [i] .executeDDL (ctx [i] .toCreateDDL (GtxTag.class));
ctx [i] .executeDDL (ctx [i] .toCreateDDL (Usr.class));
}
}
public void Div0Test () {
ctx [0] .startTrans ();
try {
new Usr (). insert (ctx [0]);
new Usr (). insert (ctx [1]);
new Usr (). insert (ctx [1]);
new Usr (). insert (ctx [2]);
System.out.println (1/0); // Force error
ctx [0] .commitTrans ();
} catch (Exception e) {
TxResult result = ctx [0] .rollbackTrans ();
GtxUnlockServ.forceUnlock (ctx [0], result);
}
Assert.assertEquals (0, ctx [0] .eCountAll (Usr.class));
Assert.assertEquals (0, ctx [1] .eCountAll (Usr.class));
Assert.assertEquals (0, ctx [2] .eCountAll (Usr.class));
}
}
The above example is the simplest demonstration of a distributed transaction. If a GTX transaction fails during the transaction submission, the final data consistency can be guaranteed regardless of whether or not part of the transaction submission occurs. Note that Gtx only supports access operations for a single entity, such as the insert / update / delete / load method, and does not support batch query or update methods such as findAll method. If the entity loaded by batch query needs to be distributed transaction management, it must be Use the load method (or exist / existStrict method check) to reload once to ensure that a rollback record is generated and a global transaction lock is created.
Note that the forceUnlock method in the above example is only used for unit testing. In the actual project, the line GtxUnlockServ.forceUnlock (ctx [0], result) should be removed and changed to use GtxUnlockServ.start (ctx, loopInterval, maxLoopQty); the method starts a separate For unlocking services, the second parameter is the unlocking interval, in seconds. It must be set to a value much greater than the database transaction timeout. The third parameter can be set to 0, indicating that there is no maximum unlocking limit.
GTX is a distributed transaction tool that supports sub-library, sub-table, and sub-library of the lock server.
public class DemoUsr extends ActiveRecord <DemoUsr> {
@Id
@ShardDatabase ({"MOD", "3"})
Integer id;
@ShardTable ({"RANGE", "10"})
Integer age;
// Getter & Setter omitted
}
public class GtxShardDbTbLockDbTest {
private static final int DB_SHARD_QTY = 5;
DbContext [] ctxs = new DbContext [3];
DbContext [] lockCtxs = new DbContext [3];
private static DataSource newTestDataSource () {
// Same as above
}
@Before
public void init () {
DbContext.resetGlobalVariants ();
DbContext.setGlobalNextAllowShowSql (true);
for (int i = 0; i <3; i ++) {
lockCtxs [i] = new DbContext (newTestDataSource ());
lockCtxs [i] .setName ("lock");
lockCtxs [i] .setDbCode (i);
lockCtxs [i] .executeDDL (lockCtxs [i] .toCreateDDL (GtxId.class));
lockCtxs [i] .executeDDL (lockCtxs [i] .toCreateDDL (GtxLock.class));
lockCtxs [i] .executeDDL (lockCtxs [i] .toCreateGtxLogDDL (DemoUsr.class));
lockCtxs [i] .setMasters (lockCtxs);
}
GtxConnectionManager lockCM = new GtxConnectionManager (lockCtxs [0]); // random choose 1
for (int i = 0; i <3; i ++) {
ctxs [i] = new DbContext (newTestDataSource ());
ctxs [i] .setName ("db");
ctxs [i] .setDbCode (i);
ctxs [i] .setConnectionManager (lockCM);
ctxs [i] .setMasters (ctxs);
ctxs [i] .executeDDL (ctxs [i] .toCreateDDL (GtxTag.class));
TableModel model = TableModelUtils.entity2Model (DemoUsr.class);
for (int j = 0; j <DB_SHARD_QTY; j ++) {
model.setTableName ("DemoUsr_" + j);
ctxs [i] .executeDDL (ctxs [i] .toCreateDDL (model));
}
}
DbContext.setGlobalDbContext (ctxs [0]); // the default ctx
}
@Test
public void commitFailTest () {
ctxs [0] .startTransOnLockDb (1);
try {
new DemoUsr (). setId (0) .setAge (0) .insert (); // locker1, db0, tb0
new DemoUsr (). setId (1) .setAge (10) .insert (); // locker1, db1, tb1
new DemoUsr (). setId (4) .setAge (11) .insert (); // locker1, db1, tb1
new DemoUsr (). setId (2) .setAge (40) .insert (); // locker1, db2, tb4
Assert.assertEquals (1, ctxs [0] .iQueryForIntValue ("select count (1) from DemoUsr_0"));
Assert.assertEquals (2, ctxs [1] .iQueryForIntValue ("select count (1) from DemoUsr_1"));
Assert.assertEquals (1, ctxs [2] .iQueryForIntValue ("select count (1) from DemoUsr_4"));
ctxs [2] .setForceCommitFail (); // Simulation DB2 transaction commit error
ctxs [0] .commitTrans ();
} catch (Exception e) {
TxResult result = ctxs [0] .rollbackTrans ();
GtxUnlockServ.forceUnlock (1, ctxs [0], result); // Unit test forces unlock server 1
}
Assert.assertEquals (0, ctxs [0] .iQueryForIntValue ("select count (1) from DemoUsr_0"));
Assert.assertEquals (0, ctxs [1] .iQueryForIntValue ("select count (1) from DemoUsr_1"));
Assert.assertEquals (0, ctxs [2] .iQueryForIntValue ("select count (1) from DemoUsr_4"));
}
}
In this example, DamoUser will be stored in different databases and sub-tables according to the id sub-key and age key. In addition, when starting a distributed transaction, you can manually specify the serial number of the current lock server, so that you can configure a lock server cluster To improve transaction processing performance, of course, in this case, the designation of the lock server is usually related to the business, such as red envelope transfer distributed transactions, the ID of the red envelope can be used as the serial number of the lock server. In actual use, GtxUnlockServ.forceUnlock (1, ctxs [0], result); no longer appears, but use the GtxUnlockServ.start (ctx, loopInterval, maxLoopQty) method to start an unlock service. For more information on the use of distributed transactions, please refer to several test examples under the unit test directory jsqlbox / function / gtx.
Finally, to add, the premise of the establishment of the above distributed transaction is that the lock server itself does not collapse, which is very important. JSqlBox is equivalent to transferring the reliability of multiple machines to the reliability of a single machine, and maintaining the reliability of a single machine is a relatively easy Mature technology, for occasions with high reliability requirements, Poxas protocol can be used to build a database cluster to ensure the reliability of the lock server.