3.2.0

New release for version 3.2.0

Feature

Click to expand

• [Feature-10683][Task Plugin] Add Java Task Plugin. (#10689) @106umao
• [DSIP-13][Feature]github_resource_plugin (#11831) @xdu-chenrj
• [Feat][UI] Set up UI setting component in profile page (#11226) @lynn-illumio
• [Feature][task-plugins] aws datasync task plugins (#11865) @Tianqi-Dotes
• [DSIP-13][python]Sub problem: gitlab, OSS and S3 resource plug-ins (#12025) @xdu-chenrj
• [python] Add resource plugin for python, dataX, CustomDataX and Sql (#12135) @xdu-chenrj
• [Feature] Optimizing data source centers (#12292) @labbomb
• [Feature-10495][Resource Center] Resource Center Refactor (#12076) @EricPyZhou
• [Feature-12040][api][ui] Add authorization management of read and write permissions for project center (#12048) @GavinGYM
• [Feature-12040][docs] Update Granted Permissions Guide for PR#12048 (#12587) @GavinGYM
• [Improvement][Task] Improved way to collect yarn job's appIds (#12197) @Radeity
• [Feature-6586][Server]add some ds process definition demo when init (#11759) @amaoisnb
• [Feature][Task] Transfer files between tasks (#12552) @jieguangzhou
• [Feature-12827] add icon in idea (#12829) @muggleChen
• Add minio as demo storage for helm charts (#12861) @kezhenxu94
• [feature][task] Add Kubeflow task plugin for MLOps scenario (#12843) @jieguangzhou
• [feat] Support set execute type to pydolphinscheduler (#12871) @zhongjiajie
• [Feature-12526][Api] add dynamic task function: shell task (#12840) @Tianqi-Dotes
• [Feature-12456][task plugin] Add new task plugin for apache linkis (#12693) @shangeyao
• [improve]Source skip check (#12900) @insist777
• [Feature][DataSource]Add trino datasource plugin (#12911) @insist777
• [Feature][Master] Run single task in workflow instance (#13103) @jieguangzhou
• Add StarRocks datasource (#13204) @ruanwenjun
• [Feature][Master] Add task caching mechanism to improve the running speed of repetitive tasks (#13194) @jieguangzhou
• [Feature][task-plugins-12820] azure azure-sql datasource plugins (#13209) @Tianqi-Dotes
• Add Terraform deploy manifests for quick setup in AWS (#13252) @kezhenxu94
• [Feature-9467] add DAMENG DataSource (#12860) @lenian
• [Feature][Deployment] Add KEDA autoscaler support for worker deployment when deployed in K8S cluster (#13367) @EricGao888
• [feature][api-datasource]add redshift auth mode to get connection by access key (#13379) @Tianqi-Dotes
• [Feature][Api] When use api to run a process we want get processInstanceId (#13184) @qianli2022
• [Feature][task]add azure datafactory task type (#13417) @Tianqi-Dotes
• Flinksql support standalone deploy mode (#13495) @shenyun
• [Fix-13434][Resource Center] fix the NPE when use OSS as the resource center (#13435) @rickchengx
• [Feature][Resource Center] One-time migration script for unmanaged resources (#13512) @Radeity
• [Feature-11195][UI] Add re-upload feature for resource files (#11203) @sketchmind
• [Feature-13331][Remote Logging] Add support for writing task logs to OSS (#13332) @rickchengx
• [Feature-13354][deploy] Add configurable busybox image in values.yaml… (#13560) @qindongliang
• [Feature-13511] Submit Spark task directly on Kubernetes (#13550) @Radeity
• [Feature][DataSource] add oceanbase datasource (#13675) @whhe
• bug: Python add online_schedule for workflow control schedule state (#13551) @zhongjiajie
• [Feature][Metrics] Tag workflow related metrics with process definition code (workflow id) (#13640) @EricGao888
• [Doc] Add OceanBase to supported list (#13700) @whhe
• [Feature-13428][Resource Center] Add support for Google Cloud Storage in the resource center (#13562) @rickchengx
• Add jdbc registry module (#13812) @ruanwenjun
• Use bootstrap user as the default tenant (#13825) @ruanwenjun
• [FEATURE][Task Plugin]Add remote-shell task plugin (#13801) @jieguangzhou
• [Improvement-13751][Worker] Support real-time pod log collection (#13752) @Radeity
• Support alert server HA (#13865) @ruanwenjun
• [Feature-13429][Remote Logging] Add support for writing task logs to Google Cloud Storage (#13777) @rickchengx
• Use percentage to represent memory/cpu usage (#13896) @ruanwenjun
• [Improvement-13824][Resource Center] User can only visit resources under tenant defaultPath (#13848) @Radeity
• [Feature][plugin] Support Kyuubi datasource (#13642) @zhaohehuhu
• [Feature][plugin] Add databend datasource in datasource plugin (#13866) @hantmac
• [Feature][Login] Add OAuth login (#8706) @Abingcbc
• [Feature] Support run all level dependent when complement (#11778) @stalary
• [Feature][plugin] Add snowflake datasource in datasource plugin (#13729) @zhaohehuhu
• [Feature-#14148][Task] Added status check interval and dependent failure policy parameters for dependent task nodes (#14150) @zhuangchong
• [Feature-14138][Metrics] Add metrics for api server (#14177) @rickchengx
• [Feature][UI] Support to view and manage all timing settings of a project. (#14178) @calvinjiang
• [Feature] Add project name/workflow definition name/instance id/task name to the built-in variables (#14099) @haibingtown
• [Feature][K8S Task] support node selector (#14126) @Radeity
• [Feature][Task Plugin] Add Dynamic task to generate dynamic worklfows with list parameters (#14127) @jieguangzhou
• [Feature][UI] Support CRUD of project-level parameters. (#14344) @calvinjiang
• [Feature-14404][datasource-plugin] add HANA datasource (#14404) @IT-Kwj
• [Feature][Api] LDAP SSL supports&& Add LDAP K8s configuration (#14349) @qingwli
• [Improvement-14318][UI] migrate version 2.x workflow definition d3 tree view to version 3.x (#14382) @yeahhhz
• [Feature-14421][K8S Task] Configurable image pull policy (#14426) @Radeity
• [Feature-14321][API] Support to complement data in descending or ascending order of date (#14450) @calvinjiang
• [Feature][Data Quality] Data Quality Support Choose Database (#14406) @qingwli
• [Feature-14542][K8S] add prometheus serviceMonitor (#14539) @Gallardot
• Support execute shell in different interceptor (#14582) @ruanwenjun
• [Feature-14505][UI] Support to set project-level preferences (#14573) @calvinjiang
• [Feature-14545][Master][UI]Cross workflow parameter passing (#14552) @Orange-Summer
• [Feature] doris add datasource (#14391) @xinxingi
• [Feature-13763][K8S Task] collect real-time log (#14379) @Radeity
• [Feature][Resource Center] Add support for Huawei Cloud OBS (#14643) @ediconss
• [RPC] Refactor the remote module. (#14776) @ruanwenjun
• [Feature][Helm] Supports deploying only partial components (#14985) @Gallardot

Improvement

Click to expand

• [Improvement-11773][api] Optimize the log printing of the api module according… (#11782) @guowei-su
• [Improvement][Test] Give an example on replacing Powermock with Mockito (#11588) @EricGao888
• [Improvement-11857][Spark] Remove the spark version of spark task (#11860) @rickchengx
• [Improvement][SQL Task]use default sql segment separator (#10869) @zhuxt2015
• [Improvement][Test] Remove powermock in dolphinscheduler-master module (#12143) @EricGao888
• [Improvement][Test] Remove powermock in service and server modules (#12164) @EricGao888
• [Improvement-12144] Remove powermock in dolphinschduler-api (#12169) @rickchengx
• [Improvement-11880][Hivecli] Improve the UI of HiveCli (#11882) @rickchengx
• [Improvement][Test] Remove the usage of powermock in datasource-plugin module (#12194) @EricGao888
• [Improvement][Test] Remove powermock from DatasyncTaskTest (#12270) @EricGao888
• [Improvement][Test] Fully remove the usage of powermock from the whole project (#12244) @EricGao888
• [Improvement][Test] Migrate all UT cases from jUnit4 to jUnit5 in task-plugin module as an example (#12299) @EricGao888
• [Improvement-12296][Test] Close static mock (#12300) @rickchengx
• [Improvement-12289][k8s] update the common.properties in values.yaml (#12290) @rickchengx
• [Migrate][Test] Migrate all UT cases from jUnit 4 to jUnit 5 in alert and api module (#12337) @EricGao888
• [improvement] Migrate all UT cases from jUnit4 to jUnit5 in dao module (#12328) @fuchanghai
• [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in datasource, registry and data-quality modules (#12352) @EricGao888
• [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in common, service, and spi modules (#12363) @DarkAssassinator
• [Improvement][Test] Block the usage of powermock and move mockito dependencies from sub-modules to root pom (#12311) @EricGao888
• [Migrate][Test] Migrate all UT cases from jUnit 4 to 5 in master, worker, and remote modules (#12362) @DarkAssassinator
• [Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @ignore to @disable (#12382) @DarkAssassinator
• [Improvement] add presto in datax (#12371) @fuchanghai
• [Migrate][Test] Add Spotless step to check and block jUnit4 imports (#12398) @EricGao888
• [Improvement][Style] Fix remaining style errors and wildcard imports (#12412) @EricGao888
• [Improvement-12333][Test] Migrate all UT cases from jUnit 4 to jUnit 5 in microbench and e2e module (#12348) @rickchengx
• [Improvement-12446][CI] Run mvn spotless apply (#12447) @DarkAssassinator
• [Improvement][Style] Update spotless junit4 check scope (#12450) @EricGao888
• [Refactor] Migrate all command-related interface functions from ProcessServiceImpl (#12474) @DarkAssassinator
• Update the project introduction (#12501) @davidzollo
• [Improvement] add springdoc-openapi-ui (#12379) @fuchanghai
• [Refactor]migrate some task dao functions from ProcessServiceImpl to dao modules. (#12505) @DarkAssassinator
• [Refactor] Migrate all workergroup-related interface functions from ProcessServiceImpl (#12493) @DarkAssassinator
• Fix alert status spelling error (#12592) @qingwli
• [Improvement-12536][k8s] Support the command for the container in k8s task plugin (#12538) @rickchengx
• [Improvement][Install] Add ${zkRoot} description. (#12596) @DarkAssassinator
• [Improvement-12623][Permission]Remove the unused projectIds in operationPermissionCheck() (#12624) @rickchengx
• Splite cpu/memory info in OSUtils#isOverload (#12663) @ruanwenjun
• [Improvement-12650][Permission] Improve the check of resourcePermissionCheck() (#12652) @rickchengx
• [Improvement][BatchQuery] Batch query ProcessDefinitions belongs to need failover ProcessInstance. (#12506) @BongBongBang
• [Improvement-12658][Permission] Improve the readability of setPerm() in grantProjectbyCode() (#12660) @rickchengx
• [Improve-5918]Remove the code logger.isInfoEnabled() (#12738) @xuchunlai
• [Document][Improvement] Add instructions on how to configure logging level for debugging purpose (#12699) @EricGao888
• [Bug] [dolphinscheduler-aop] aspectj-maven-plugin version undefined (#12756) @Radeity
• [Improvement-12335][ui] Make the resources to be reactive in hivecli task (#12336) @rickchengx
• [typo] Update use-dependent.ts typo (#12462) @moonkop
• [Improvement][script] Add installPath verification for install.sh (#12523) @muggleChen
• [Improvement-12029][dolphinscheduler-api] Upgrade the snakeyaml version to avoid vulnerabilities (#12726) @seedscoder
• Upgrade clickhouse jdbc driver (#12639) @simsicon
• [Improvement][CI] Update github mergeable label regex (#12815) @EricGao888
• [Improvement-12850][UT] Improve the ut of DataSourceMapperTest (#12851) @rickchengx
• [Improvement-12838][UT] Improve the ut of datasource (#12839) @rickchengx
• Use System#lineSeparator to replace /n (#12889) @ruanwenjun
• [Improvement-12856][UT] Refactor duplicated code fragment in UdfFuncMapperTest (#12857) @huage1994
• Fix master cluster may loop command unbalanced (#12891) @ruanwenjun
• [Document] Fix spelling (#12935) @jbampton
• Fix grammar and spelling (#12937) @jbampton
• [Improvement-12853][UT] Improve the ut of ResourcePermissionCheckServiceTest (#12854) @rickchengx
• [Improvement-12804][UT] Remove the unused method in DataAnalysisControllerTest (#12805) @rickchengx
• [Feature-10498] Mask the password in the log of sqoop task (#11589) @rickchengx
• [Bug-12956] fix incorrect java path (#12957) @zhangfane
• [Improvement-12907] Change heartbeat log level to debug (#12980) @DarkAssassinator
• [Fix-12916] Add permission check when query or download log (#12917) @rickchengx
• [Improvement-12763][UT] Improve testListAuthorizedProjects() in ProjectMapperTest (#12768) @rickchengx
• Set max loop times when consume StateEvent to avoid dead loop influence the thread. (#13007) @ruanwenjun
• Don't source env file in task, this will override tenant's environment (#13028) @ruanwenjun
• fix typo error in YarnClientAspect (#13047) @Radeity
• Print GC datestamps in gc log (#13059) @ruanwenjun
• [Improvement][Style] Expand all static imports and add CI check to block wildcard static imports (#13067) @EricGao888
• [improvement-#11879]Keyword Search Improvement (#12951) @fuchanghai
• Add plugin-all module (#13079) @ruanwenjun
• Update dolphinscheduler_ddl.sql - for PostgreSQL (#13081) @darrkz
• [Improvement][Alert] Prevent default alert group from binding alert plugin instances with id 1 and 2 automatically (#13114) @EricGao888
• [Improvement] optimization task definition & fix in the task definition list, if one task have more pre task, the task list can't show all task (#13106) @jackfanwan
• Revert "Update dolphinscheduler_ddl.sql - for PostgreSQL" (#13117) @ruanwenjun
• [Fix] Fix Token E2E Bug (#13069) @qingwli
• [Improvement] Fixed Possible JSON ordering permutations problem in Tests (#13118) @bmk15897
• [Improve-13001]migrate commons-collections -> commons-collections4 (#13002) @DarkAssassinator
• Format task parameter as pretty json (#13173) @ruanwenjun
• Remove logic task plugin module (#13171) @ruanwenjun
• [Improvement][ApiServer]Task defination list paging sorted by updateTime not effect (#12622) @tracehh
• [Improvement][Metrics] Filtering data by application in grafana-demo worker dashboard example (#13139) @xuchunlai
• [Improve] Create v2folder (#13192) @insist777
• [refactor] cache compiled regex pattern (#13208) @youzipi
• [Improvement][api-sql] Optimize the sql for paging query project (#13172) @hiSandog
• Remove datasource in procedure task (#13198) @ruanwenjun
• [Improvement][Master] fix spell error and move comment to correct describe location(issue 13232) (#13233) @ZhongJinHacker
• Fix spelling (#13237) @jbampton
• Remove dao module in worker (#13242) @ruanwenjun
• [Improvement][api]Batch query for optimizing query process-list interface (#13222) @hiSandog
• Use mybatis in mysql registry module (#13275) @ruanwenjun
• Remove the log path check (#13280) @ruanwenjun
• [Improvement][docker] Add SPRING_JACKSON_TIME_ZONE to .env file (#13271) @hiSandog
• Remove alert when delete workflow instance (#13281) @ruanwenjun
• [Improvement][dao] Optimized countInstanceStateByProjectCodes (#13273) @hiSandog
• Add projectCode in t_ds_process_instance and t_ds_task_instance to avoid join (#13284) @ruanwenjun
• [Improvement][Master] Validate same content of input file when using task cache (#13298) @Radeity
• Add source env list in common.properties (#13317) @ruanwenjun
• Delete workflow will delete workflow instance (#13336) @ruanwenjun
• [Fix-13341][Improvement][UI] Worked out the issue with the long name overflowing on the table. (#13342) @calvinjiang
• rename: WorkflowDefinitionTree component rename (#13134) @zhoufanglu
• Clear the task group data when delete a project/workflowInstance (#13383) @ruanwenjun
• Use spotless upToDateChecking to improve the performance. (#13394) @ruanwenjun
• Remove entity in QueryWrapper constructor (#13396) @ruanwenjun
• Add documentation to release Helm Chart (#13385) @kezhenxu94
• Change remove task instance log to async request (#13399) @ruanwenjun
• Use parallelStream to improve the performance of upgrade (#13442) @ruanwenjun
• Task instance failure when worker group doesn't exist (#13448) @ruanwenjun
• [Refactor][UI]Add a back button when viewing an error file (#13354) @imizao
• Change mysql COLLATE to utf8_bin (#13469) @ruanwenjun
• Use convert to set the exist column collate to utf8_bin (#13479) @ruanwenjun
• Use single thread to refresh kerberos (#13456) @ruanwenjun
• Fix task log might delay until task finished due to batch flush (#13502) @ruanwenjun
• [fix-13483] Fix complement data date order will change due to use set (#13487) @dddyszy
• [Doc] Update LDAP Doc (#13507) @qingwli
• [Improvement-13472][Api] Removes the overhead of idle Quartz Threadpool of Api (#13475) @xxjingcd
• [Improvement][task node] task node resources improvement (#13508) @zhutong6688
• Fix the parameter table is messed up (#13526) @kezhenxu94
• [Improvement-13491][*] Use lombok @slf4j annotation to generate logger (#13509) @seedscoder
• Alter title in t_ds_alert to 512 (#13535) @ruanwenjun
• Fix typos in tools for demo (#13538) @Abingcbc
• [Improvement] add cost time for api call (#13518) @caishunfeng
• [fix-13504] Display the current project name on the left corner of the page (#13572) @QuantumXiecao
• [Improvement] Support modal close for datasources (#13586) @devosend
• [improve-#13201] update pid during running (#13206) @fuchanghai
• [Improvement-13437][UT] Add UT for S3StorageOperator (#13530) @rickchengx
• Add execute function to handle the workflow instance operation (#13610) @ruanwenjun
• [Improvement-12782][UT] Add UT for the 13 implementations of ResourceAcquisitionAndPermissionCheck (#12784) @rickchengx
• [Improvement-12697][UT] Improve the ProjectServiceTest (#12698) @rickchengx
• Fix task instance log order might be wrong due to async print (#13660) @ruanwenjun
• Add missing placeholder (#13663) @Radeity
• [Improvement][master-sql] Optimize the sql for queryTheHighestPriorityTasks (#13671) @hiSandog
• [Improvement][api] Remove unless code in ResourcesServiceImpl (#13676) @hiSandog
• Use MDC to filter task instance log (#13673) @ruanwenjun
• [improve-#13665] improve datax parser if sql contain * (#13680) @fuchanghai
• Clearer task runnable. (#13689) @ruanwenjun
• Ignore unknown VM options in start.sh (#13719) @ruanwenjun
• remove duplicate dependencies (#13773) @NoSuchField
• [Improvement-13651] Remove useless resources task record (#13721) @Radeity
• Modify name len to 255 (#13788) @ruanwenjun
• Fix retry task instance will loss varpool (#13791) @ruanwenjun
• Take over task instance in master failover (#13798) @ruanwenjun
• Refactor remote command (#13809) @ruanwenjun
• [Improvement][Dependent] support self-dependent (#13818) @caishunfeng
• [Improvement-13653][Remote Logging] Support api-server to get task log from remote target (#13654) @rickchengx
• Use common response for sync request (#13847) @ruanwenjun
• Use DATAX_PYTHON to specify a datax python version (#13849) @ruanwenjun
• [Improvement]Update spelling mistake for the name of WorkflowSubmitStatue (#13850) @SYSU-Coder
• Fix trim might cause NPE in PropertyUtils (#13853) @ruanwenjun
• Remove the extra default tenant in workflow save page (#13867) @ruanwenjun
• [Improvement-13884][UI] Improve the UI of flink sql task (#13885) @rickchengx
• [Improvement-13911][JSON] Improve the readability of workflow exported json files (#13912) @rickchengx
• Display user binding tenant if exist (#13909) @ruanwenjun
• [Improvement][Task Plugin] Integrate zeppelin password authentication with DS zeppelin task plugin (#13861) @xiaomin0322
• [Improvement-13932][API] Tenant parameters and Worker group parameters are put together in the task startup parameters (#13935) @zhuangchong
• [Improvement-13785] upgrade zk and pg image version to support arm architecture (#13789) @eye-gu
• [Improvement-13804] Optimal HDFS configuration acquisition (#13804) (#13811) @kingbabingge
• [improve-#13966] mask post jdbc info in datax log (#13969) @fuchanghai
• [improvement-#14010] resource file manager page,repeat onMounted optimize (#14011) @ly109974
• Refactor logic task (#13948) @ruanwenjun
• [Improvement][API Test] Add API tests for worker group related APIs (#13936) @EricGao888
• [Improvement-14057][api] alarm instance add sort (#14058) @ly109974
• [Improvement-14052][Log] Remove the useless filter in getAppIdsFromLogFile (#14053) @rickchengx
• [Improvement-14054][Worker] Kill multiple yarn apps at the same time (#14055) @rickchengx
• [Improvement-14095][Resource Center] Implement getResourceFileName in StorageOperator (#14097) @Radeity
• chore: roadmap should not label stale (#14103) @zhongjiajie
• [Improvement-13868][K8S Task] Support the custom labels for k8s task (#14067) @Radeity
• Add supported operations of workflow definition (#14122) @rickchengx
• [Improvement-14112][UI] Add a quick link to visit workflow instances of the current workflow. (#14134) @calvinjiang
• Fix task exception might not exist in task instance log (#14085) @ruanwenjun
• [Improvement-14083][Resource Center] Handle deleteSource in OSS / S3 / GCS (#14084) @rickchengx
• fix: PULL_REQUEST_TEMPLATE.md dead link (#14163) @Radeity
• fix: disordered comments in values.yaml (#14197) @Radeity
• [Improvement-14201][UI] Improve the select component to support users to filter options. (#14202) @calvinjiang
• [Improvement-14195][Parameter] remove misleading comment of combining local and global parameter (#14196) @rickchengx
• Remove state check for dependent/subProcess in StateWheelExecuteThread (#14242) @ruanwenjun
• [Improvement][Metrics] Add more worker related metrics and fix some previous ones (#14254) @EricGao888
• [Improvement-14286][BOM] Bump clickhouse-jdbc from 0.3.2-patch11 to 0.4.6 (#14287) @rickchengx
• [Improvement] Add baseDao for CRUD (#14316) @ruanwenjun
• [fix-14123] remove the equals and hashcode in Project pojo (#14167) @Rianico
• [Improvement][Etcd] Support SSL In Etcd And Enhance Etcd In Helm (#13924) @qingwli
• [Improvement-14331][common] Remove the duplicate ParameterUtils (#14332) @rickchengx
• [Improvement][API-Test] Add api tests for project page (#14402) @EricGao888
• revert: Datasource need test binding to create prod (#14381) @zhongjiajie
• [Improvement-14311][Permission]Remove duplicate checkProjectAndAuth (#14313) @rickchengx
• [Improvement-14395][UI] Description input box can not input blank (#14396) @Radeity
• [Improvement-11913] Mask password when creating/editing alert instances (#14415) @c3Vu
• [Improvement-14393][Resource Center] Remove deprecated resource description in api and UI page (#14394) @Radeity
• chore: Add new step skip for ui build and spotless (#14427) @zhongjiajie
• [Improvement-14437][UI] Fixed the issue with failing to save the task for the HiveCLI task. (#14438) @calvinjiang
• [Improvement-14371][Hivecli] Add support for hivevar setting in hivecli task (#14423) @rickchengx
• [Improvement-14457][UT] Improve the expected and actual value of Assertions (#14459) @rickchengx
• [Improvement][API Test] Add API test cases for process definition controller (#14300) @EricGao888
• [Improvement][API Test] Add api test cases for scheduler page (#14480) @EricGao888
• [Improvement][API] Fix some nits in apis (#14481) @EricGao888
• [document]Modify the inconsistencies of grammar and wording in README_zh_CN.md (#14471) @ahuljh
• [Improvement-14456][UI] When making the process online the system should remind user to take care of its offline timing. (#14460) @calvinjiang
• [improvement] DataQuality module improve (#14463) @boy-xiaozhang
• [Improvement][Document] Fix worker failover service comment (#14494) @EricGao888
• Use jvm_args_env.sh to set JVM args (#14487) @ruanwenjun
• Polish config, expose rpc config in application.yml (#14501) @ruanwenjun
• [Improvement-14387][UI] Support to reset user's password. (#14498) @calvinjiang
• [Improvement-12700][Project]Improve the sql of listAuthorizedProjects (#12702) @rickchengx
• [Improvement][API-Test] Add api tests for executor controller (#14500) @EricGao888
• ci: Require up to date before merging (#14530) @zhongjiajie
• [Improvement][API Test] Add api tests for process instance (#14514) @EricGao888
• Add WorkflowExecuteContext (#14544) @ruanwenjun
• [Typo] standardized license header. (#14548) @FlechazoW
• [Improvement][Task] Mask configYaml in task log (#14571) @Gallardot
• [Improvement] Add log for get local NetworkInterface (#14578) @ruanwenjun
• [Dependency] Bump h2, snowflake-jdbc, snappy-java version (#14575) @SbloodyS
• [Improvement-14136][task] Support submitting spark-sql task with the sql file in resource center (#14527) @rickchengx
• [Improvement-14587][UI] Improve the environment component (#14588) @calvinjiang
• Expose swagger.properties to release (#14590) @ruanwenjun
• impv: Correct datax execute and python execute script name (#14523) @zhongjiajie
• [Improvement] Dataquality code style enhance (#14592) @qingwli
• Add host/port in heartbeat (#14591) @ruanwenjun
• [Improvement][K8S] Disable delete namespaces in the K8S cluster (#14597) @Gallardot
• [Improvement-14269][API] Bind task group with project (#14392) @Radeity
• [Improvement][HELM] add driver-class-name (#14614) @Gallardot
• [Improvement-4375][api] cannot delete yarn queue (#13046) @amaoisnb
• [Improvement][CI] deploy to k8s (#14555) @Gallardot
• [Feature] Support filter restrict network interface (#14638) @ruanwenjun
• [Improvement] Support get property from env/jvm/common.properties (#14645) @ruanwenjun
• [Improvement][UI] Image pull policy in the K8S task provides default values (#14650) @Gallardot
• [Workflow] Print WorkflowInstance detail/ TaskInstance detail when finished (#14677) @Mukvin
• [Improvement][helm] configmap changes automatically trigger restart (#14657) @Gallardot
• Fix message in MessageRetryRunner might disorder (#14725) @ruanwenjun
• Upgrade JDK17 for Sonar (#14726) @ruanwenjun
• [Fix-13972][dao] set default value for command (#14612) @eye-gu
• [Improvement][UI] Support to disable or enable the project preferences. (#14756) @calvinjiang
• [Improvement] add dispatch fail times when compare (#14761) @caishunfeng
• Add rpc benchmark test (#14797) @ruanwenjun
• [Improvement-14658][etcd] Distinguish between add and update by using pre kv (#14659) @eye-gu
• [Improvement-14835][UI] Support to immediately adjust the fill color of the task node after changing the run flag of the task. (#14837) @calvinjiang
• [Feature-14802][api] Use Casdoor SSO to log in, add admin user configuration (#14814) @gaopeng666
• [Improve] Fix placeholders should match arguments in log statement (#14850) @lamberken
• [Improvement-14870][UI] Optimize task disable node style in dark theme (#14871) @yeahhhz
• [Improvement][API] Support to check if the worker group has been used by any tasks or schedulers when users delete or rename it. (#14893) @calvinjiang
• [Improvement-14884][Master] add overload state in heartbeat to trigger slot change (#14887) @Radeity
• [Improvement-14916][dolphinscheduler-alert] alert message contains the taskPriority (#14919) @chenshuai1995
• [Improvement-14894][dolphinscheduler-alert] alert message contains the operator (#14910) @chenshuai1995
• Set charset in upgrading tools to solve read upgrade sql mojibake (#14955) @ruanwenjun
• [Bug] [Api] Logback file STDOUT duplicate (#14966) @qingwli
• [Improvement] Fix typo for redshift datasource (#14973) @EricGao888
• [feature-#14449] add secret in k8s Task (#14930) @fuchanghai
• [Improvement][Common] Obtain resource information in the k8s environm… (#14968) @winghv
• Fix task log appender will not be closed immediate after throwing exception (#14989) @ruanwenjun
• [Improvement][Task] Mask password in task log (#14988) @Gallardot
• [Improvement][K8S] K8S task support passing context parameters to downstream task (#14934) @Gallardot
• [Improvement][Alert] HTTP Related Alert plugin add retry logic (#14972) @qingwli

Bugfix

Click to expand

• [Improve] Improve support test tasks UT (#12062) @insist777
• Fix update TaskDefinition error (#12060) @ruanwenjun
• [fix#12195] Change date to timestamp to support cross diff time zones (#12239) @DarkAssassinator
• [Bug][Registry] Fix mysql registry doesn't work due to mysql doesn't support setArray (#12255) @ruanwenjun
• [improve] Correct some log in scheduler (#12278) @fuchanghai
• [Bug] [Service] Use MapUtils to judge globalParamsMap is empty (#12384) @hstdream
• Fix #11236, source is not available in sh (#12180) @simsicon
• [fix#12364][UI] fix when creating a workflow, click Data Quality or Machine Learning to expand both (#12365) @pandong2011
• [FIX][DAO-12277] change t_ds_fav_task column task_name to task_type and related code (#12276) @Tianqi-Dotes
• [Fix-12109] Fix the errors when starting 2 times with dolphinscheduler-daemon.sh (#12118) @rickchengx
• [Bug] fix same status code definition (#12724) @Radeity
• fix: Fix some mistakes in the datasource (#12775) @labbomb
• [fix-12687]when timeout_notify_strategy is null, modify timeout_notify_strategy (#12689) @jackfanwan
• [bug][alert-12865] translate alert input field tips to chinese when using chinese (#12879) @Tianqi-Dotes
• [fix] Add token as authentication for python gateway (#12893) @zhongjiajie
• [fix-#12932] when subprocess's processInstance is fail,not notify parent processInstance (#12933) @fuchanghai
• [Bug] [DOC] fix doc lead CI failed (#12964) @jieguangzhou
• [fix] Python gateway can not upload to resource center (#13042) @zhongjiajie
• [ci] Update skywalking-eye to fix block CI (#13076) @zhongjiajie
• [Bug] Fix task log file might not be clear (#13102) @ruanwenjun
• Optimize event loop to fix the event might not be handle since workflow finished (#13193) @ruanwenjun
• [Bug][Document] Fix a tiny bug in release doc (#13239) @EricGao888
• [Improvement][Master] Get varPool from upstream tasks instead of processInstance if dag is incomplete (#13255) @jieguangzhou
• Revert "[Bug] [Server] Once click online schedule time, task will be automatically scheduled" (#13295) @ruanwenjun
• [Improvement][Worker] Startup parameter should have the highest priority (#13274) @Radeity
• [Fix-13338] [API] Fix that when the timing data is not configured with environmental information, the timing management does not display the data (#13339) @zhuangchong
• [Bug][TaskPlugin] Task execution error occurs when the Zeppelin task zeppelinNoteId parameter value is an empty string (#13162) @LucasClt
• [Bug][Deployment] Fix minio version in helm chart (#13368) @EricGao888
• [doc] Remove excess space (#13388) @LiXuemin
• [fix-12721] Fix cannot modify the upstream task in task definition page. (#12722) @jackfanwan
• Split the upgrade code (#13436) @ruanwenjun
• [Fix-12828][api] Add permission check when query specific datasource (#12830) @rickchengx
• FIX: 3.1.1_schema mysql ddl sql error (#13485) @stalary
• [bug][UI]Fixed a bug where the environment name of a task could not be changed when a workflow was created(issue-12457) (#13148) @hdygxsj
• [Improvement][Task Plugin] Support HTTP Task Plugin output response as a parameter (#13529) @jieguangzhou
• [BUG][Resource Center] NPE caused by uninitialized hdfsProperties (#13540) @Radeity
• [improve-#13597] fix the method name (#13599) @fuchanghai
• [fix-13603][DATAX]SQLServer not support special field (#13604) @fuchanghai
• [fix-13625] Modify systemd-run from MemoryMax to MemoryLimit for more generality. (#13627) @QuantumXiecao
• docs(file-manage): The file in File manage is a relative path, not an absolute path (#13641) @iuhoay
• Fix-13708 [Bug] [Resouces] Resource file search paging error (#13709) @zhutong6688
• [doc] update kubernetes doc (#13716) @Radeity
• [Bug-13782] match ')' in setValue regular expression (#13787) @Radeity
• [fix-13734] [api] Fixed the problem that the corresponding task definition could not be queried after workflow import correctly(#13734) (#13735) @sandiegoe
• fix: Alarm instance port verification (#13827) @zhongjiajie
• [fix] DB2 username can not be save (#13764) @AliceXiaoLu
• [Fix-13802][UI] Display english column name in resource center (#13806) @Radeity
• [Bug-13882][Worker] Upgrade kubernetes-client version to 6.0.0 (#13891) @Radeity
• [BUG][Resource Center] Task can not select main JAR (#13536) @Radeity
• [Fix-13820][alert] Fix bodyParams is empty get error log (#13821) @kingbabingge
• [Fix-13950][master] fix the key of master node failover (#13952) @eye-gu
• [Fix-13977][SQL] 3.2.0_schema mysql ddl sql error (#13979) @Radeity
• [fix-#13956] [Master]taskId is null cause NPE (#13980) @fuchanghai
• [Bug][Worker] get worker status bug (#13992) @Radeity
• [Bug][API] update processInstance bug (#13991) @Radeity
• [Bug-13929][K8S Task] Set command contains single quote is invalid (#13930) @Radeity
• Fix registry table field (#14043) @ruanwenjun
• FIX: allLevelDependent default value (#14059) @stalary
• [Fix-13989] Add path validation (#14021) @ly109974
• [Fix-14049][dao] fix init sql and use on conflict do update in pg (#14051) @eye-gu
• [CI] Try fix api test error (#14094) @jieguangzhou
• [Bug-14105][Task] Replace default tenant when downloading resource (#14106) @Radeity
• [Fix-14117][DAO] Ironed out a lot of syntax errors in the file of 3.2.0_schema/postgresql/dolphinscheduler_ddl.sql (#14124) @calvinjiang
• [Bug][ui] fix error TS2339: Property $route does not exist on type CreateComponentPublicInstance (#14144) @zhuangchong
• [Fix-14140] [API] Fix dependent task nodes in workflow instances cannot display data (#14141) @zhuangchong
• Revert [fix-#13956] [Master]taskId is null cause NPE #13980 (#14142) @zhuangchong
• [Fix-13381][plugin] fix error while parsing hive load sql (#13378) @alextinng
• [Bug][Worker] generate kill command with unknown user: default (#14145) @Radeity
• [fix-14119] The checkBucketNameExists method should not use s3Client. listBuckets() (#14188) @Zzih
• Fix master memory leak due to MasterTaskExecuteRunnableDoesn't be removed (#14162) @ruanwenjun
• Fix taskGroupQueue doesn't removed from inQueue where wakeup failed (#14200) @ruanwenjun
• [Improvement-14146][K8S Task] print complete error logs in task log file (#14147) @Radeity
• fix: trim node selector requirement value (#14307) @Radeity
• [Fix-14334][master] Fix the complement function does not generate the next running instances in parallel mode (#14335) @zhuangchong
• Fix switch task use boolean type error (#14326) @ruanwenjun
• [Bug-14378][Master] Trigger task execution in workflow instance page cause workflow instance keep running status (#14389) @Radeity
• oss avoid listBuckets permission in bucket check (#14414) @zhangkuantian
• [Bug][Doc] Fix dead link in api test docs (#14419) @EricGao888
• [Fix] Fix running task instance throught api gots failed (#14433) @qingwli
• [Fix-14488][UI] When creating a tenant add the strong validation of its tenant code. (#14492) @calvinjiang
• [Fix-13987][bom] set 3.17.2 version protobuf-java (#13998) @eye-gu
• [Bug][Core] Fix query process instances by trigger code bug (#14528) @EricGao888
• fix terrafrom config bug (#14362) @zhongjiajie
• [Fix-14503][Worker] Fix the problem of subprocess cannot be killed (#14504) @rickchengx
• [fix-14517][api] An error occurred while trying to access the Postgres data… (#14574) @hoey94
• [Fix-14531][UI] Tooltip of Chart not show text in light theme mode (#14532) @arlendp
• [Bug][dolphinscheduler-api]validated for duplicate username when creating or registering a user (#14577) @Gallardot
• Support parse ipv6 (#14584) @ruanwenjun
• [Fix-14603] phone Number validate (#14604) @pppppjcc
• [Fix-14601][UI] Ironed out the issue with choosing an environment on the task form of both types of DMS and DATASYNC. (#14605) @calvinjiang
• [Fix-13976][dao] task complete contain force success state (#14611) @eye-gu
• [Fix-14512][BOM] kubernetes-client version conflict (#14561) @Radeity
• [Fix-14566][master] Fix serial_wait command_params does not replace global_params (#14606) @CallMeKingsley97
• [Fix-10795][plugin] sqoop task query columns bug (#10776) @CallMeKingsley97
• [Fix-14675][Alert]batchInsert execption (#14676) @Gallardot
• [Fix-14668][dolphinscheduler-tools] Modify the imported jar package. Use org.apache.commons.lang3.StringUtils. (#14669) @alei1206
• [Fix-14685] [Task] Remove override method k8sConfigYaml in ShShellInterceptorBuilder (#14682) @Radeity
• [Fix-14686][Alert]batchInsert execption (#14687) @Gallardot
• [Fix-14696] [Startup scripts] /bin/java command Argument list too long (#14697) @SbloodyS
• [Doc]fix typo (#14691) @Gallardot
• [Fix-14683][pom] upgrade mysql driver version (#14684) @eye-gu
• Fix serial mode will cause NPE in Workflow bootstrap (#14703) @ruanwenjun
• [Fix-14714][master] package task instance before put into cache (#14718) @eye-gu
• [Fix-14715] [Resource Center] Modify default fs of local storage plugin (#14716) @Radeity
• Set tenant as the the resource file owner (#13832) @ruanwenjun
• fix fault tolerant enviroment missing (#14735) @jieguangzhou
• [Fix-14721] [K8S Task] Handle job delete event when user manually delete the job in k8s cluster (#14722) @Radeity
• fix: kubectl version update (#14762) @Gallardot
• fix: null app-info-path cause NPE (#14752) @Radeity
• Fix cache is overriden randomly after build and before tests (#14808) @kezhenxu94
• [Fix-14819][Master] Fix the full log path of logical task is not be set when task is dispatched. (#14821) @guowei-su
• [Fix-14820][Master] Fix the status of logical task is not be correctly changed when task is finished. (#14822) @guowei-su
• Add workflowInstanceHost and taskInstanceHost in ITaskInstanceExecutionEvent (#14827) @ruanwenjun
• [Fix-14828][master] check task id not null to avoid NPE (#14829) @eye-gu
• [Fix-14149][master] fix workflow always ready pause (#14161) @eye-gu
• fix invalid registry path ends with 'master' or 'worker' (#14328) @zinw
• [Bug] [UI] New tasks with the flag normal show a grey color (#14862) @qingwli
• Fix session/thread leak at ssh datasource (#14859) @ruanwenjun
• [Fix-14660] [Resource Center] Remove NONE storage type and set LOCAL type as default (#14661) @Radeity
• [Fix-14811][api] Fix when using Casdoor SSO, ldap, OAuth2 to login, the tenant problem of not specifying the currently logged in user (#14812) @gaopeng666
• [Bug][Worker] Fix sql/shell task doesn't replace parameter (#14876) @zhongjiajie
• [Fix-14790][dolphinscheduler-ui] Fix the data quality task, keyword filtering not supported (#14792) @chenshuai1995
• Fix java.sql.Date serialize error by convert it to java.util.Date (#13050) @JueLance
• [Bug][Worker]Global and local parameters are mismatch in SQL task (#14926) @Gallardot
• [fix-#14918] add missing ddl (#14921) @fuchanghai
• Fix the ephemeralId will not be deleted before jdbc-registry closed. (#14936) @dong6349
• [Fix-14895][API] Keep printing null if task log file not exist (#14912) @Radeity

Document

Click to expand

• Remove sensitive configs from dolphinscheduler_env.sh (#11917) @kezhenxu94
• [Improvement-#11608][task-plugin] New time variables are added to facilitate business development (#11667) @fengjian1129
• [Feature][doc] Add a guide to nominate new committer (#12175) @TyrantLucifer
• [feat][ci] Add pre-commit for easy spotless config (#12192) @zhongjiajie
• [Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) (#11868) @jieguangzhou
• [doc] create oracle help.md (#12218) @Chris-Arith
• [Doc][BUG] Documentation bug lead to CI error #12559 (#12560) @jieguangzhou
• [chore] Separate Python API into another repository (#12779) @zhongjiajie
• [fix][doc] Dead link for python api (#12848) @zhongjiajie
• [Feature-8030][docs] Add sqoop task doc (#12855) @baihongbin
• [Improvement][Task Plugin] Improvement Kubeflow task plugin (#12928) @jieguangzhou
• docs: update faq, fix spelling, remove repeated words (#12934) @jbampton
• [Document] Fix grammar and spelling (#12940) @jbampton
• [doc][datasource-12820] add azure sql database doc (#13254) @Tianqi-Dotes
• [Improvement][Document] Fix minor spelling and punctuation errors (#13452) @EricGao888
• [feat] Add type local turn on resource center by default (#13303) @zhongjiajie
• [doc] spark on native k8s (#13605) @Radeity
• docs: update rainbond docs (#13608) @zzzhangqi
• [Improvement]update spelling mistake for Constants (#13738) @tracehh
• doc: Announce link error to zh (#13855) @zhongjiajie
• [Improvement][DOC] Update resource S3 configuration docs (#13985) @jieguangzhou
• chore: Release version 3.0.6 (#14234) @zhongjiajie
• Remove code owner (#14461) @kezhenxu94
• [Feature-14251][Task] Support yarn queue definition in yarn task (#14310) @Radeity
• [Fix-14907]Fix click to view the history in the task node of the workflow instance (#14909) @zhuangchong
• [Doc]fix typo (#15005) @Gallardot

Chore

Click to expand

• [Improvement-12293] Update the common.properties in api-test-case and e2e-case (#12295) @rickchengx
• [fix-#12875]command line that -Dmaven.test.skip=true have no effect (#12876) @fuchanghai
• [chore][python] Change name from process definition to workflow (#12918) @zhongjiajie
• [ci] Require docs context before merge, and run spotless check (#12977) @zhongjiajie
• [hotfix][ci] Remove dead link check path filter (#12985) @zhongjiajie
• [Improvement] delete star chart (#13953) @davidzollo
• [Improvement-14080][script] cleanup dist directory at the beginning of check-LICENSE.sh (#14081) @rickchengx
• chore: Remove EOL issue version and add prepare one (#14270) @zhongjiajie
• [CI] Try to fix flaky CI (#14568) @SbloodyS
• Disable dependabot pull requests (#14596) @kezhenxu94
• [CI] [E2E] Increase container healthcheck timeout (#14599) @SbloodyS
• [CI] [Chore] Increase browser container timeout (#14608) @SbloodyS
• [DISCUSS]ci: Mergeable must have ready-to-merge and not require version (#14600) @zhongjiajie
• [CI] [E2E] Increase e2e containers timeout (#14628) @SbloodyS
• [Chore] [E2E] Add e2e optional for ci (#14633) @SbloodyS
• [CI][E2E] Upgrade e2e dependency version (#14634) @SbloodyS
• chore: Correct db upgrade and add ci avoid regression (#14620) @zhongjiajie

Others

Click to expand

• [feature][API]feature: add task type list and enable user to add task type to fav (#11727) @Tianqi-Dotes
• Fix alert_send_status may throw duplicate key exception and add limit for query alert (#11953) @ruanwenjun
• [Feature-11473][Task]Support test task (#11670) @insist777
• refactor: dag-menu refactor (#11965) @labbomb
• [Improvement][Python API] Validate version of Python API and Java gateway at launch (#11626) @hezean
• [test] fix flaky test in CI (#12017) @zhongjiajie
• [improvement] improve relation between notification strategy and alarm group on ui (#12200) @fuchanghai
• Reduce the size of tarball to continue ASF release (#15004) @zhongjiajie