Skip to content

Commit

Permalink
Merge branch 'apache:dev' into fix-flink-2pc-runmode-set
Browse files Browse the repository at this point in the history
  • Loading branch information
@zhilinli123
zhilinli123 authored Aug 8, 2023
2 parents baeb4b1 + f4aa219 commit 0ff1ba4
Show file tree
Hide file tree
Showing 88 changed files with 3,089 additions and 354 deletions.
13 changes: 8 additions & 5 deletions .asf.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,21 @@
#

github:
description: SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).
description: SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
homepage: https://seatunnel.apache.org/
labels:
- data-integration
- change-data-capture
- cdc
- high-performance
- offline
- real-time
- data-pipeline
- sql-engine
- batch
- streaming
- data-ingestion
- apache
- seatunnel
- etl-framework
- EtLT
- elt
enabled_merge_buttons:
squash: true
merge: false
Expand Down
3 changes: 1 addition & 2 deletions docs/en/connector-v2/sink/DB2.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,7 @@ semantics (using XA transaction guarantee).
| support_upsert_by_query_primary_key_exist | Boolean | No | false | Choose to use INSERT sql, UPDATE sql to process update events(INSERT, UPDATE_AFTER) based on query primary key exists. This configuration is only used when database unsupport upsert syntax. **Note**: that this method has low performance |
| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete. |
| max_retries | Int | No | 0 | The number of retries to submit failed (executeBatch) |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `batch_interval_ms`<br/>, the data will be flushed into the database |
| batch_interval_ms | Int | No | 1000 | For batch writing, when the number of buffers reaches the number of `batch_size` or the time reaches `batch_interval_ms`, the data will be flushed into the database |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `checkpoint.interval`<br/>, the data will be flushed into the database |
| is_exactly_once | Boolean | No | false | Whether to enable exactly-once semantics, which will use Xa transactions. If on, you need to<br/>set `xa_data_source_class_name`. |
| generate_sink_sql | Boolean | No | false | Generate sql statements based on the database table you want to write to |
| xa_data_source_class_name | String | No | - | The xa data source class name of the database Driver, for example, DB2 is `com.db2.cj.jdbc.Db2XADataSource`, and<br/>please refer to appendix for other data sources |
Expand Down
8 changes: 1 addition & 7 deletions docs/en/connector-v2/sink/Jdbc.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
| connection_check_timeout_sec | Int | No | 30 |
| max_retries | Int | No | 0 |
| batch_size | Int | No | 1000 |
| batch_interval_ms | Int | No | 1000 |
| is_exactly_once | Boolean | No | false |
| generate_sink_sql | Boolean | No | false |
| xa_data_source_class_name | String | No | - |
Expand Down Expand Up @@ -107,12 +106,7 @@ The number of retries to submit failed (executeBatch)

### batch_size[int]

For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `batch_interval_ms`
, the data will be flushed into the database

### batch_interval_ms[int]

For batch writing, when the number of buffers reaches the number of `batch_size` or the time reaches `batch_interval_ms`
For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `checkpoint.interval`
, the data will be flushed into the database

### is_exactly_once[boolean]
Expand Down
3 changes: 1 addition & 2 deletions docs/en/connector-v2/sink/Mysql.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,7 @@ semantics (using XA transaction guarantee).
| support_upsert_by_query_primary_key_exist | Boolean | No | false | Choose to use INSERT sql, UPDATE sql to process update events(INSERT, UPDATE_AFTER) based on query primary key exists. This configuration is only used when database unsupport upsert syntax. **Note**: that this method has low performance |
| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete. |
| max_retries | Int | No | 0 | The number of retries to submit failed (executeBatch) |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `batch_interval_ms`<br/>, the data will be flushed into the database |
| batch_interval_ms | Int | No | 1000 | For batch writing, when the number of buffers reaches the number of `batch_size` or the time reaches `batch_interval_ms`, the data will be flushed into the database |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `checkpoint.interval`<br/>, the data will be flushed into the database |
| is_exactly_once | Boolean | No | false | Whether to enable exactly-once semantics, which will use Xa transactions. If on, you need to<br/>set `xa_data_source_class_name`. |
| generate_sink_sql | Boolean | No | false | Generate sql statements based on the database table you want to write to |
| xa_data_source_class_name | String | No | - | The xa data source class name of the database Driver, for example, mysql is `com.mysql.cj.jdbc.MysqlXADataSource`, and<br/>please refer to appendix for other data sources |
Expand Down
3 changes: 1 addition & 2 deletions docs/en/connector-v2/sink/OceanBase.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,7 @@ Write data through jdbc. Support Batch mode and Streaming mode, support concurre
| support_upsert_by_query_primary_key_exist | Boolean | No | false | Choose to use INSERT sql, UPDATE sql to process update events(INSERT, UPDATE_AFTER) based on query primary key exists. This configuration is only used when database unsupport upsert syntax. **Note**: that this method has low performance |
| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete. |
| max_retries | Int | No | 0 | The number of retries to submit failed (executeBatch) |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `batch_interval_ms`<br/>, the data will be flushed into the database |
| batch_interval_ms | Int | No | 1000 | For batch writing, when the number of buffers reaches the number of `batch_size` or the time reaches `batch_interval_ms`, the data will be flushed into the database |
| batch_size | Int | No | 1000 | For batch writing, when the number of buffered records reaches the number of `batch_size` or the time reaches `checkpoint.interval`<br/>, the data will be flushed into the database |
| generate_sink_sql | Boolean | No | false | Generate sql statements based on the database table you want to write to |
| max_commit_attempts | Int | No | 3 | The number of retries for transaction commit failures |
| transaction_timeout_sec | Int | No | -1 | The timeout after the transaction is opened, the default is -1 (never timeout). Note that setting the timeout may affect<br/>exactly-once semantics |
Expand Down
Loading

0 comments on commit 0ff1ba4

Please sign in to comment.