WIP: feat(merge): merge main to feat/424_taskframework by krihy · Pull Request #1310 · oceanbase/odc

krihy · 2023-12-27T01:51:26Z

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Additional documentation e.g., usage docs, etc.:

…g_add_odc_monitor # Conflicts: # server/odc-server/src/main/resources/log4j2.xml # server/odc-service/src/main/java/com/oceanbase/odc/service/systemconfig/DbConfigChangeMatcher.java

…nitor

…#987) * fix: test masking * fix: unit test

Co-authored-by: IL MARE <yh263208@oceanbase.com>

* add DataX dependency * add datax.jar * use Hikari to replace Druid * currently remove ThroughputMonitor * use plugins to replace accessor * fix pattern * clean up files * remove redundant params * format code * remove decrease count * resolve conflicts * fix file pattern * fix sorted bug * log summary in finally block * fix UT failures * change file regex * use ScheduledExecutorService instead of 'new Thread' * call datax by process * fix a bug * fix UT * remove unused variable * support configure jvm params for datax * response to CR comments --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

* fix there is no min,hor,sec in date * fix issue #937 * fix type error * fix unit test error

* fix: not clear inactive connection * fix: create connectionConfig by built-in admin user in bastion * refine * fix: ut * fix ut and refine * fix * fix: H2 SQL syntax error

…and mysql mode (#1001) Co-authored-by: IL MARE <yh263208@oceanbase.com>

* add ServletConfiguration to set properties of filters * set dispatcherServlet property --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

* fix: project role name i18n * refine

…iple sqls during sql interception stage and pre-check stage (#976) * stash * adapt oboracle * adapt SqlSplitter * fix * refacto * adapt * fix ut cases * adapt to iterator * add log * efficiency opt * code format * fix * refactor ut * delete unused method * update submodule * response to comments * fix * bugfix * fix * Revert "fix" This reverts commit 6aa6eba. * adapt * fix * fix NPE * delete config * response to comments --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

… names (#1011) * fix: horizontal unauthorized * code format * fix ut --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

…uplicated columns (#1021)

…ole (#1019) * fix: permission * delete useless codes

…ponse (#1039) * fix: parse response * fix: ut * fix: ut

…roject (#968) * fix: block internal database when auto-sync * code foramt * fix: allow transfer blocked databases

* update submodule * refactor rule applying * init default rule applyings * update submodule * stash * stash * fix typo * delete config * delete unused code * add ut case * add ut cases

* fix drop check rule * fix sql check's npe * add inteceptor order * fix failed to locate issue line * fix unit test error

#1053) * fix mysql1479 get partition * delete empty line

* add bilk save for nodes * add test cases * make bind flow element batch * create instance node on building topo * fix unit test * merge yiang's code * add several repo * use batch to insert entity * add batchUpdate into OdcJpaRepository * use batchCreate method * self test fix * make shortUniqueId transient --------- Co-authored-by: ungreat <ungreatang@gmail.com>

* fix: apply project task print log error * fix: execution deadline time display failed

…ror message (#1233)

…eating a data source in team space (#1243)

… "pending approval" tickets (#1260)

… ob-mysql (#1253) * adapt to the dialect type of frond end donot include ODP sharding mysql * response to review

…1266) * close resource * format * review * update submodule --------- Co-authored-by: yh263208 <yh263208@oceanbase.com>

#1227) * upgrade mock version * upgrade mock version * optimize log printing * make driver compatible to ojdbc 8 * add npe * add data mock compress method * optimize log printing * remove useless comments * fix issue #189 * supports mysql80 * add concurrent config * fix internation * control front end's mock limit * add max output size limit * refact err msg * change version and remove useless code * supports lowercase env * upgrade submodule * increase max count of mock data

* supports get log for DLM job. * supports get log for DLM job. * supports get log for DLM job. * supports get log for DLM job. * supports get log for DLM job. * update path pattern * code format * Remove redundant code * destroy thread pool after task finished. * check task parameters when update schedule task. * init cluster name. * fix NPE when variables are not exist. * Verify whether the data source is a cluster instance * Supports configuring connection timeout. * Supports configuring sharding strategy. * modify default sharding strategy. * Remove redundant code * fix create dlm job failed in MySql mode. * Fix edit limit config and description not taking effect * fix interrupt task failed. * check job status before update. * fix restart job failed. * fix restart job failed. * fix restart job failed. * fix restart job failed. * format * update submodule commit id * do not check cluster name when create job. * support configuring scan batch size. * merge 4.2.x to current branch. * change default sharding strategy. * code format * response comments * fix sharding strategy configuration didn't work. * upgrade sdk version * fix ut * fix description of key * update default sharding strategy.

* fix mock data charset error * optimize code

* fix druid log * rollback

* add change log * fix type err * response to cr comments * update submodule * update changelog * add odp to changelog * update changelog * add change log. * response to cr comments * response to cr comments * response to cr comments * response to cr comments * add eng change log * update change log * fix type error * remove dot * remove lfs --------- Co-authored-by: tinker <guowenlong.gwl@oceanbase.com>

…ace (#1274) while use spring security configure straightly, the CsrfFilter refer invalid session strategy firstly of AccessDeniedHandler in CsrfFilter. new the CsrfFiler manual, make csrf token lifecytle decouple from session manage, to avoid CsrfToken error while logout from private space

* set columns for result set export task * use url.toURI() instead of url.getFile() * rename method * set schema for connectionInfo * fix result export failed * create dir if not exists * rename MySQLWriterPluginParameter#Connection * fix failed to mask data for mysql result set export task

…SQL in some special cases (#1280)

* fix failed to rewrite sql for mysql * fix not skip header for export * only add as for native mysql * sql format always requires header * fix an error * fix no content for table name in Chinese * avoid NPE

* adjust sql console rule default values * fix

* upgrade sdk version to 1.0.8 * upgrade sdk version to 1.0.8

…ion (#1294) * refact max data count of mocking * update submodule * upgrade ob-data-mocker's version * delete unused properties

…led by default (#1302)

merge dev/4.2.x into main

ungreat and others added 30 commits November 27, 2023 17:18

add ODCMonitor

3efbdf9

add ODCMonitor

0f3a319

rename

3efaf48

Merge remote-tracking branch 'origin/yiang_add_odc_monitor' into yian…

b0e59c1

…g_add_odc_monitor # Conflicts: # server/odc-server/src/main/resources/log4j2.xml # server/odc-service/src/main/java/com/oceanbase/odc/service/systemconfig/DbConfigChangeMatcher.java

fix(connection):add back connection cluster name (#942)

7d70d0f

Merge remote-tracking branch 'origin/dev/4.2.x' into yiang_add_odc_mo…

e205022

…nitor

rename

2120a18

add log message

a90d0dc

add log message

ac0d9b6

fix(data-security): test masking algorithm may lead to security issue (…

10104b7

…#987) * fix: test masking * fix: unit test

Merge branch 'dev/4.2.x' into yiang_add_odc_monitor

60d7d38

feat(osc): lock user is not required when create osc task on ob (#970)

710938a

Co-authored-by: IL MARE <yh263208@oceanbase.com>

Merge branch 'dev/4.2.x' into yiang_add_odc_monitor

5dbbd31

fix(variables): variable updating may lead to sql injection (#1008)

920de2e

* fix there is no min,hor,sec in date * fix issue #937 * fix type error * fix unit test error

fix(bastion): inactive datasources are not cleared (#997)

00712de

* fix: not clear inactive connection * fix: create connectionConfig by built-in admin user in bastion * refine * fix: ut * fix ut and refine * fix * fix: H2 SQL syntax error

fix(db-browser): DB session list show 0 in execute time for ob mysql …

8f73554

…and mysql mode (#1001) Co-authored-by: IL MARE <yh263208@oceanbase.com>

feat(sql-execute): unable to obtain locale info in subthread (#994)

0e6b909

* add ServletConfiguration to set properties of filters * set dispatcherServlet property --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

fix(apply-project): project role names are not internationalized (#1000)

80f3632

* fix: project role name i18n * refine

fix(ticket): horizontal unauthorized when query approver related role…

95b390c

… names (#1011) * fix: horizontal unauthorized * code format * fix ut --------- Co-authored-by: IL MARE <yh263208@oceanbase.com>

fix(data-security): create sensitive columns failed due to scanning d…

651e7a5

…uplicated columns (#1021)

fix(permission): user can create datasource without any project and r…

67d2642

…ole (#1019) * fix: permission * delete useless codes

fix(integration): cannot deal with array when parsing json or xml res…

2c488af

…ponse (#1039) * fix: parse response * fix: ut * fix: ut

fix(database): block built-in databases when auto-sync databases to p…

3658888

…roject (#968) * fix: block internal database when auto-sync * code foramt * fix: allow transfer blocked databases

fix(sql-rules): cannot add/update any sql rule default values (#1014)

df479e7

* update submodule * refactor rule applying * init default rule applyings * update submodule * stash * stash * fix typo * delete config * delete unused code * add ut case * add ut cases

review

030b793

fix(sql-check): failed to recognize several drop statements (#1026)

e09137f

* fix drop check rule * fix sql check's npe * add inteceptor order * fix failed to locate issue line * fix unit test error

rename

85c589e

fix(partition-plan): create partition plan task failed in obmysql 1479 (

dd40713

#1053) * fix mysql1479 get partition * delete empty line

ungreat and others added 28 commits December 19, 2023 14:35

feat(monitor):add api alarm (#1212)

1febbcb

fix(apply-project): failed to set mdc value (#1237)

131347e

* fix: apply project task print log error * fix: execution deadline time display failed

fix(web-framework): invalid csrf token result into Invalid session er…

eeb6f7a

…ror message (#1233)

fix(rollback-plan): NPE when user input sql content is empty (#1242)

2169f3e

fix(datasource): it occurs 'duplicate data source name' error when cr…

f2fcebc

…eating a data source in team space (#1243)

fix(partition plan):failed to disable table partition plan (#1247)

ad3999d

fix(schema-plugin): show partition name with identifiers (#1249)

66b45de

fix(monitor): format druid log (#1251)

d14ddea

fix(ticket): project "pending approval" tickets shows other project's…

3e10bf6

… "pending approval" tickets (#1260)

fix(datasource): convert the type of ob-mysql-sharding data source to…

7055e86

… ob-mysql (#1253) * adapt to the dialect type of frond end donot include ODP sharding mysql * response to review

fix(flow): close prepared stmt and resultset when batch creating end (#…

b8f9fd5

…1266) * close resource * format * review * update submodule --------- Co-authored-by: yh263208 <yh263208@oceanbase.com>

fix(mock-data): failed to recognize the charset key of 'UTF8' (#1272)

f8c41d7

* fix mock data charset error * optimize code

fix(monitor): format alarm error stack to inline (#1273)

cac37e4

* fix druid log * rollback

fix(datasource): make ODP_SHARDING_OB_MYSQL not be converted to OB_MY…

608a866

…SQL in some special cases (#1280)

fix(result-set-export): failed to rewrite sql for mysql (#1288)

51788c2

* fix failed to rewrite sql for mysql * fix not skip header for export * only add as for native mysql * sql format always requires header * fix an error * fix no content for table name in Chinese * avoid NPE

fix(sql-rule): adjust several sql-console rules' default values (#1281)

3c0fd8b

* adjust sql console rule default values * fix

feat(dlm): upgrade dlm's version to 1.0.8 (#1299)

c2d53cd

* upgrade sdk version to 1.0.8 * upgrade sdk version to 1.0.8

feat(mock-data): increase the max number of the mock data to 100 mill…

d85b83a

…ion (#1294) * refact max data count of mocking * update submodule * upgrade ob-data-mocker's version * delete unused properties

fix(sql-rule): the rule 'allow-sql-types' of dev environment is disab…

5ef3a9b

…led by default (#1302)

builds: update submodule (#1304)

c543368

build: fix copy obclient logic (#1305)

09e4ec8

merge dev/4.2.x into main #1308

a310761

merge dev/4.2.x into main

yhilmare assigned krihy Dec 27, 2023

krihy closed this Dec 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

WIP: feat(merge): merge main to feat/424_taskframework#1310

WIP: feat(merge): merge main to feat/424_taskframework#1310
krihy wants to merge 113 commits intofeat/424_taskframeworkfrom
main

krihy commented Dec 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Comments

Conversation

krihy commented Dec 27, 2023

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Additional documentation e.g., usage docs, etc.:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants