Skip to content

2.3.9 Release

Latest
Latest
Compare
Choose a tag to compare
@TyrantLucifer TyrantLucifer released this 19 Jan 15:20
· 38 commits to dev since this release
2.3.9
074a02e

2.3.9 Release Note

Bug fix

  • [Bug][Avro] Fix avro miss convert short type to int (#8459)
  • [Bug][CI] Fix all-connectors and update-modules run simultaneously (#8343)
  • [Bug] [connector-file] When the data source field is less than the target (Hive) field，it will throw null pointer exception#8150 (#8200)
  • [Bug][connectors-v2] fix mongodb bson convert exception (#8044)
  • [Bug] [connectors-v2] The Hadoop Source/Sink fails with Unable to find valid Kerberos Ticket. (#7809)
  • [Bug][Connector-v2] MongoDB CDC Set SeatunnelRow's tableId (#7935)
  • [Bug][Core] Local mode cancel running JobStatusRunner (#7943)
  • [bugfix][core] Fix the problem of incorrect association between metrics and nodes (#7799)
  • [BUG] Fixed avro format support for storing null (#8424)
  • [BugFix][Excel] Fix read formulas/number cell value of excel (#8316)
  • [Bug][Kafka] kafka reads repeatedly (#8465)
  • [Fix][API] Fix column length can not be long (#8039)
  • [Fix][CI] Fix ci dependency check error (#8262)
  • [Fix][CI] Fix CI error caused by set classloader-cache-mode to true (#7940)
  • [Fix][CI] Fix schedule test be canceled (#8024)
  • [Fix][CI] time-consuming module optimization (#8052)
  • [Fix][Connecotr-V2] Fix clickhouse sink does not support composite primary key (#8021)
  • [Fix][Connector-starrocks] Fix drop column bug for starrocks (#8216)
  • [Fix][Connector][TDEngine] TDEngine support NCHAR type (#8411)
  • [fix][connector-v2][connector-assert] Optimize Assert Sink verification method (#8356)
  • [Fix][Connector-V2][connector-file-base-hadoop] Fixed HdfsFile source load the krb5_path configuration (#7870)
  • [Fix][Connector-V2] Fix AbstractSingleSplitReader lock useless when do checkpoint (#7764)
  • [Fix][Connector-V2] Fix cdc use default value when value is null (#7950)
  • [Fix][Connector-V2] Fixed clickhouse connectors cannot stop under multiple parallelism (#7921)
  • [Fix][Connector-V2] Fix file binary format sync convert directory to file (#7942)
  • [Fix][Connector-V2] Fix hana type loss of precision (#7912)
  • [Fix][Connector-V2] Fix hive krb5 path not work (#8228)
  • [Fix][Connector-V2] Fix kafka format_error_handle_way not work (#7838)
  • [Fix][connector-v2]Fix Paimon table connector Error log information. (#7873)
  • [Fix][Connector-V2] Fix When reading Excel data, string and date type conversion errors (#7796)
  • [Fix][Connector-V2][FTP] Fix FTP connector connection_mode is not effective (#7865)
  • [Fix][Connector-V2]Oceanbase vector database is added as the source server (#7832)
  • [fix][docker] update Dockerfile (#8421)
  • [Fix][Docker] Workflow "push-docker" failed run with error "No space left on device" (#8339)
  • [Fix][Doris] Fix catalog not closed (#8415)
  • [fix] fix get job metrics get NPE (#8093)
  • [Fix][Flink] Fix NPE when get null row from upstream transform (#8469)
  • [Fix] [Flink/Spark] Fix transform stream choose wrong when not configure source_table_name (#7907)
  • [Fix][Kafka] Fix in kafka streaming mode can not read incremental data (#7871)
  • [fix][log]Fix log returns interface issue and adds test code (#8387)
  • [fix] move docker build plugin into dist module (#8338)
  • [Fix] Set all snappy dependency use one version (#8423)
  • fix transforms-v2 nlpmodel typo (#8430)
  • [Fix][Zeta] Fix apply resource again for another pipeline when restore one pipeline (#7965)
  • [Fix][Zeta] Fix job client print error msg even job finished (#7981)
  • [Fix][Zeta] Fix worker node metrics acquisition (#7862)
  • [Fix][Zeta] Introduce SeaTunnel CompletableFuture to prevent ForkJoinPool thread shortage (#8445)
  • [Fix][Zeta] Remove duplicate LIFECYCLE_WRITER_CLOSE events (#7984)
  • [Fix][Zeta] Set AsyncLogger.ThreadNameStrategy=UNCACHED to avoid thread name cache (#8215)
  • [Hotfix][CDC] Fix occasional database connection leak when read snapshot split (#7918)
  • [Hotfix][Config] Fix configuration key sort disorder (#7893)
  • [hotfix] [connector-cdc-oracle ] support read partition table (#8265)
  • [hotfix][connector-elasticsearch-sink] Convert index to lowercase (#8429)
  • [hotfix][connector-v2][hive]Fix hive3 e2e dependencies (#8045)
  • [Hotfix][Connector-V2][kafka] fix kafka sink config exactly-once exception (#7857)
  • [Hotfix][Core][Flink] SeaTunnel flink engine support application mode on yarn (#7762)
  • [Hotfix][Docs]Correct errors in the FAQ doc (#8048)
  • [Hotfix][E2E] Fix OracleCDCIT compile error (#8023)
  • [HotFix]Fix Dead Link In ObsFile.md (#8251)
  • [hotfix] fix exceptions caused by operator priority in connector-clickhouse when using sharding_key (#8162)
  • [Hotfix] Fix redis sink NPE (#8171)
  • [hotfix][gbase] Fix gbase driver link (#8110)
  • [Hotfix][Transform][Sql] function concat_ws handle array type， avoid write [Ljava.lang.String. (#8369)
  • [Hotfix][Transforms-v2] DynamicCompile Plugin compatibility fix (#8057)
  • [Hotfix][Zeta] Fix the dependency conflict between the guava in hadoop-aws and hive-exec (#7986)
  • [Hotfix][Zeta][Log] Fix zeta parallelStream log trace for job (#8456)

Docs

  • [Chore] delete chinese desc in code (#8306)
  • [Core][Flink] Resolve streaming source high CPU usage (#8354)
  • [doc][connector][hive] Add hive chinese doc (#8169)
  • [Doc][Format] Add protobuf doc (#7989)
  • [Doc][hive] Rename result_table_name to plugin_output in hive doc (#8203)
  • [Docs] [Elasticsearch] fix the doc about index name replacement (#8392)
  • [Doc][Transform-v2] Update doc for Array function (#8419)
  • [Fix][Doc] Correct hive-jdbc config useKerberos to use_kerberos (#7896)
  • [Fix][Doc] fix backup-count config (#8147)
  • [Fix][Doc] fix doc tip style and add document format specification (#8378)
  • [Fix][Doc] Fixed unreachable links (#8131)
  • [Fix][Doc] Fixing errors in the documentation and adding information (#8353)
  • [Fix][Doc] Fix jetty doc error (#7883)
  • [Fix][Doc] Fix key description error in redis zh doc (#7993)
  • [Fix][Doc] Fix klaviyo doc 404 link (#7969)
  • [Fix][Doc] Fix outdated plugin path in oracle documentation (#8403)
  • [Fix][Doc] Fix rest-api missing content (#8384)
  • [Fix][Doc] Fix wrong link in about/index.html (#8486)
  • [Fix][Doc] Fix zh doc build error (#7882)
  • [Fix][Doc] Modify error name SEATNUNNEL to SEATUNNEL (#8257)
  • [Fix][Doc] Modify the example of enable-dynamic-port in the documentation. (#8331)
  • [Fix][docs] fix wrong config in setup.md (#8483)
  • [Fix][Doc] Update Elasticsearch Sink and Oracle CDC Source Sample Configuration (#8425)
  • [Improve][doc] Add doc name check (#8140)
  • [Improve][Doc] Add enable_header_write parameter in File doc (#8397)
  • [Improve][Doc] Add file_filter_pattern example to doc (#7922)
  • [Improve][Doc] Add some connector icons (#7944)
  • [Improve][Doc] Add the way of configure jvm options in client (#8091)
  • [Improve][Doc] Correcting typo errors in logging (#7874)
  • [Improve][Doc] Correction to docker doc for get container ip (#7801)
  • [Improve][Doc] Fix chinese doc miss grafana dashborad config (#7955)
  • [Improve][Doc] Improve the connector-v2 develop doc (#8190)
  • [Improve][Doc] Move seatunnel engine deployment doc (#8239)
  • [Improve][Docs] Add driver reference to doris/starrocks docs (#8414)
  • [Improve][Docs] Edit the description about startup.mode for Opengauss-CDC (#7815)
  • [Improve][Doc] Update connector v2 contribute guide (#8375)
  • [Improve][Doc] Updated telemetry doc (#7827)

Feature

  • [Feature][API] Support timestamp with timezone offset (#8367)
  • [Feature][CDC] Add 'schema-changes.enabled' options (#8252)
  • [Feature][CDC] Add 'schema-changes.enabled' options (#8285)
  • [Feature]Check Chinese comments in the code (#8319)
  • [Feature][ClickhouseFile] Support add publicKey to identity (#8351)
  • [Feature][Clickhouse] Support sink savemode (#8086)
  • [Feature][connector-elasticsearch] elasticsearch support nested type (#8462)
  • [Feature] [connector-milvus] update milvus connector to support dynamic schema, failed retry, etc. (#7885)
  • [Feature] [Connector-Redis] Redis connector support delete data (#7994)
  • [Feature][Connectors] LocalFile Support reading gz (#8025)
  • [Feature][Connector-V2] Add prometheus source and sink (#7265)
  • [Feature][Connector-V2[Doris]Support sink ddl (#8250)
  • [Feature][Connector-V2]Ftp file source support multiple table (#7795)
  • [Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 (#7840)
  • [Feature][Connector-V2] Jdbc DB2 support upsert SQL (#7879)
  • [Feature][Connector-V2] Piamon Sink supports changelog-procuder is lookup and full-compaction mode (#7834)
  • [Feature][Connector-V2]Sftp file source support multiple table (#7824)
  • [Feature][Connector-V2] Starrocks implements multi table sink (#8467)
  • [Feature][Connector-V2] StarRocks-sink support schema evolution (#8082)
  • [Feature][Connector-V2]Support Doris Fe Node HA (#8311)
  • [Feature][Connector-V2] Support MaxCompute save mode (#8277)
  • [Feature][connector-v2]Support opengauss jdbc connnector using opengauss driver. (#7622)
  • [Feature][Connector-v2] Support S3 filesystem of paimon connector (#8036)
  • [Feature][Connector-v2] Support schema evolution for Oracle connector (#7908)
  • [Feature][Connector-v2] Support schema evolution for paimon sink (#8211)
  • [Feature][Connector-V2] Support the jdbc connector for highgo db (#8282)
  • [Feature][Connector-V2] Support use EasyExcel as read excel engine (#8064)
  • [Feature][Connector-V2] Support write cdc changelog event into hudi sink (#7845)
  • [Feature][Core] Rename result_table_name/source_table_name to plugin_input/plugin_output (#8072)
  • [Feature][Core] Support cdc task ddl restore for zeta (#7463)
  • [Feature][Core] Support read arrow data (#8137)
  • [Feature][Doc]Paimon Souce changelog update (#8187)
  • [Feature][Doris] Support multi-table source read (#7895)
  • [Feature][E2E] Add E2E test case time analysis (#8028)
  • [Feature][E2E] Add hive3 e2e test case (#8003)
  • [Feature][E2E][Kerberos] Support for Kerberos in e2e (#8108)
  • [Feature][Elasticsearch] Support sink ddl (#8412)
  • [Feature][Engine UI] partial pages of the engine (#7602)
  • [Feature][File] Support config null format for text file read (#8109)
  • [Feature][Iceberg] Support custom delete sql for sink savemode (#8094)
  • [Feature][Jdbc] Add Jdbc default dialect for all jdbc series database without dialect (#8132)
  • [Feature][JDBC source] pg support char types (#8420)
  • [Feature][Jdbc] Support sink ddl for postgresql (#8276)
  • [Feature][K8S deploy] add helm chart (#8291)
  • [Feature][Kafka] Add debezium_record_table_filter and fix error (#8391)
  • [Feature][Mongodb-CDC] Support multi-table read (#8029)
  • [Feature][MySQL-CDC] Support database/table wildcards scan read (#8323)
  • [Feature][Redis] Flush data when the time reaches checkpoint.interval (#8198)
  • [Feature][Redis] Flush data when the time reaches checkpoint.interval and update test case (#8308)
  • [Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)
  • [Feature] [rest-api] Support Rest Api to upload file and submit task (#8442)
  • [Feature][RestAPI] Support submit job with seatunnel style hocon format config (#8000)
  • [Feature][Rest api] Use jetty to replace Hazelcast native rest api (#7647)
  • [Feature][Transform-SQL]Support sql transform to generate UUID (#7881)
  • [Feature][Transform] Sql transform support array and split function (#8016)
  • [Feature][Transform] Support merge shading-tables to one table (#8360)
  • [Feature][Transform] Support rename table/column (#8170)
  • [Feature][transforms-v2] Support append only stream from cdc source (#7763)
  • [Feature][transform] transform support explode (#7928)
  • [Feature][Transform-v2] Add metadata transform (#7899)
  • [Feature][Transform-V2] Spark support transform with multi-table (#8340)
  • [Feature][Transform-V2] Support transform with multi-table (#7628)
  • [Feature][Zeta] Add COS support for checkpoint storage (#7931)
  • [Feature][Zeta]Add jar path precheck when job submit on master (#7976)
  • [Feature][Zeta][Core] Support rest api get logs (#7818)
  • [Feature] [Zeta] Optimize CoordinatorService ThreadPool Configuration to Prevent Potential OOM (#8241)
  • [Feature][Zeta] Submit job scheduling support pending (#7693)
  • [Feature][Zeta] Support delete logs regularly (#7787)
  • [Imporve][ClickhouseFile] Directly connect to each shard node to obtain the corresponding path (#8449)
  • [Imporve][ClickhouseFile] Improve clickhousefile generation parameter configuration (#8293)
  • [Improve] Allow nullable for SeaTunnelRowType convert to Column (#8098)
  • [Improve][api] Improve inheritance relationship of AlterTableColumnEvent (#7825)
  • [Improve][API] Move AlterTableNameEvent parent (#7869)
  • [improve][API] support metrics interface in rest v2 (#8312)
  • [Improve][Api] Supports simultaneous config of conditions and optional options (#7788)
  • [Improve][API] Unified tables_configs and table_list (#8100)
  • [Improve][CI]add code specification check (#8209)
  • [Improve][CI] Fix module runs CI repeatedly (#8284)
  • [Improve][CI] Improve oracle schema evolution e2e case (#8224)
  • [Improve][CI] Optimizing CI time cost (#7997)
  • [Improve][CI]skip ui module, improve module dependent (#8225)
  • [Improve][ClickhouseFile] Added attach sql log for better debugging (#8315)
  • [Improve][ClickhouseFile] ClickhouseFile Connector's rsync transmission supports specifying users (#8236)
  • [Improve][ClickhouseFile] Clickhouse supports option configuration when connecting to shard nodes (#8297)
  • [Improve][ClickhouseFile Connector] Unified specifying clickhouse file generation path (#8302)
  • [Improve][ClickhouseFile] Improve rsync log output (#8332)
  • [Improve][connector-e2e][Hive]Optimize script and method names (#8249)
  • [Improve][Connector-file-base] Improved file allocation algorithm for subtasks. (#8453)
  • [Improve][Connector][Hive] skip temporary hidden directories (#8402)
  • [Improve][Connector-V2] Add doris/starrocks create table with comment (#7847)
  • [Improve][Connector-V2] Add pre-check for table enable cdc (#8152)
  • [Improve][Connector-V2] Add pre-check starrocks version before exeucte alter table field name (#8237)
  • [Improve][Connector-V2] Add some debug log when create dir in (S)FTP (#8286)
  • [Improve][Connector-V2] Add table info when multi table sink error (#8163)
  • [Improve][Connector-V2] Change File Read/WriteStrategy setSeaTunnelRowTypeInfo to setCatalogTable (#7829)
  • [Improve][Connector-V2] Fix SqlServer cdc memory leak (#8083)
  • [Improve][Connector-V2][FTP] delete duplicated code (#7915)
  • [Improve][Connector-V2] Improve jdbc merge table from path and query when type is decimal (#7917)
  • [Improve][Connector-V2] Improve schema evolution on column insert after for mysql-jdbc (#8017)
  • [improve][connector-v2][kafka-connector] Add e2e code for Kafka Kerberos authentication (#8159)
  • [Improve][Connector-V2] Optimized DefaultSaveModeHandlerconstruct method (#8179)
  • [Improve][Connector-V2] Redis support custom key and value (#7888)
  • [Improve][Connector-V2] Reduce the create times of iceberg sink writer (#8155)
  • [Improve][Connector-V2] Reduce the request times of iceberg load table (#8149)
  • [Improve][Connector-V2] RocketMQ Sink add message tag config (#7996)
  • [Improve][Connector-V2] Save data savemode operation when schema savemode created new table (#8178)
  • [Improve][Connector-v2] Support checkpoint in batch mode for paimon sink (#8333)
  • [Improve][Connector-V2] Support read comment when jdbc dialect without catalog (#8196)
  • [Improve][Connector-V2] The interface supports jdbc respects the target database field type (#8031)
  • [Improve][Connector-v2] Use checkpointId as the commit's identifier instead of the hash for streaming write of paimon sink (#7835)
  • [Improve][Connector-v2] Use regex to match filedName placeholders in jdbc sink (#8222)
  • [Improve][Core] Add protobuf transform test case (#7914)
  • [Improve][Core] Add test case to check deprecated result_table_name and source_table_name (#8153)
  • [Improve][Core] Refactor common options of column/row (#7911)
  • [Improve][dist]add shade check rule (#8136)
  • [Improve][dist]add UT class name check (#8182)
  • [Improve][E2E] improve doris initialize (#8446)
  • [Improve][E2E] Improve Druid E2E Case (#8077)
  • [Improve][E2E] improve kafka e2e (#8295)
  • [Improve][E2E] improve oracle e2e (#8292)
  • [Improve][E2E] modify the method of obtaining JobId (#7880)
  • [Improve][Elasticsearch] Truncate the exception message body for request errors (#8263)
  • [Improve][Example] Add log config for job mdc (#7876)
  • [Improve][Example] Improve zeta local/cluster example (#7877)
  • [Improve][Examples]Enable Flink Example WebUI (#8355)
  • [Improve][Excel] Support read blank string & auto type-cast (#8111)
  • [Improve][Fake] Improve memory usage when split size is large (#7821)
  • [Improve][Formats] Support not primary-key table for debezium format (#7836)
  • [Improve][Format] Support complex data type parse of debezium_json (#8330)
  • [Improve][Iceberg] Filter catalog table primaryKey is empty (#8413)
  • [Improve][Iceberg] Support table comment for catalog (#7936)
  • [Improve] Improve AlterTableSchemaEventHandler (#8050)
  • [improve] improve hazelcast joiner, lite node can't be election as master (#8261)
  • [Improve][Jdbc] Improve ddl write validate (#8158)
  • [Improve][Jdbc] Optimize index name conflicts when create table for postgresql (#7875)
  • [Improve][Jdbc] Refactor ddl change (#8134)
  • [Improve][Jdbc] Support nvarchar in dm (#8270)
  • [Improve][Jdbc] Support postgresql inet type. (#7820)
  • [Improve][Jdbc] Support save mode for the sink of jdbc-dm (#7814)
  • [Improve][Kafka] Support custom topic for debezium compatible format (#8145)
  • [Improve][POM] Bump the version of maven-shade-plugin to 3.4.1 (#8067)
  • [Improve][PostgreSQL CDC]-PostgresSourceOptions description error (#7813)
  • [Improve] Remove unused code (#8167)
  • [Improve][Rest api]add verifyHtmlResponse test case (#7919)
  • [Improve][rest api] improve rest-api doc (#7804)
  • [Improve][RestApi] Unify the code logic of REST API V1 and V2 (#7920)
  • [Improve][RestApi] Update context-path to / as default path (#7800)
  • [Improve][Sls] Add sls sink connector、e2e、doc (#7830)
  • [Improve][Test] Enable some spark test case on jdk 11 (#8238)
  • [Improve][Transform] Auto mark column length with jsonpath transform (#7636)
  • [Improve][Transform] gz support excel (#8181)
  • [Improve][Transform] Optimize transform structure (#8071)
  • [Improve][Transform] Rename sql transform table name from 'fake' to 'dual' (#8298)
  • [Improve][transform-v2] Support dynamic types for array function (#8139)
  • [Improve] update clickhouse connector, use factory to create source/sink (#7946)
  • [Improve] Update snapshot version to 2.3.9 (#7841)
  • [Improve] Update snapshot version to 2.3.9 (#7849)
  • [Improve][Zeta] Fixed issues related to log deletion during ci (#8076)
  • [Improve][Zeta] Jetty server enable dynamic port (#7903)
  • [Improve][Zeta] Make sure the local mode CompletableFuture behavior same as server mode (#8476)
  • [Improve][Zeta] Make the default value of classloader-cache-mode is true (#7938)
  • [Improve][Zeta] Remove default jvm memory option (#8068)
  • [Improve][Zeta] Split classloader in config parse phase (#8193)
  • [Improvment][Build] Provide option to disable source jar creation. (#7680)
  • [Zeta][Log] Fix trace parallel stream (#8206)
Assets 2
Loading