【2.3.9】【又来BUG了】mysql到starrocks跑批，source加上过滤条件，结果读不到数据，导致写SR不生效

[andyuan1000]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

1.加上过滤条件source打印为空，不生效
2.实际上query有数据的，导致插入不了SR数据

SeaTunnel Version

2.3.9

SeaTunnel Config

#定义一些作业的运行参数,具体可以参考 https://seatunnel.apache.org/docs/2.3.1/concept/JobEnvConfig
env {
  job.mode="BATCH"  #作业的运行模式，BATCH=离线批同步，STREAMING=实时同步
  job.name="SeaTunnel_market_new_58_163_sr_Job"
  checkpoint.interval=10000 #每10000ms进行一次checkpoint，后面会详细介绍checkpoint对JDBC Source和StarRocks Sink这两个连接器的影响
  read_limit.bytes_per_second=7000000
  read_limit.rows_per_second=400
}
source {
  Jdbc {
    plugin_output="Table9210050164000"
    connection_check_timeout_sec = 1000
    parallelism=5 # 并行度，这里是启动5个Source Task来并行的读取数据
    partition_column="id" # 使用id字段来进行split的拆分，目前只支持数字类型的主键列，而且该列的值最好是离线的，自增id最佳
    partition_num="10" # 拆分成20个split，这20个split会被分配给5个Source Task来处理
    query="SELECT `id`,record_time, `order_code`, `status`, `create_date`, `order_total_price`, `order_pay_mode`, `old_order_code`, `actual_money`, `operate_flag`, `order_commodity_weight`,cabinet_id, `cabinet_name`, `cabinet_code`,order_status FROM `market_new2`.`t_electronic_back_order` where date(create_date) > '2025-01-20' "
    password="hdic211"
    driver="com.mysql.cj.jdbc.Driver"
    user=root
    url="jdbc:mysql://8.34.53.26:23306/market_new2?useUnicode=true&characterEncoding=utf-8&autoReconnect=true&zeroDateTimeBehavior=convertToNull&useSSL=false"
  }
}
transform {
  # 在本次示例中我们不需要做任务的Transform操作，所以这里为空，也可以将transform整个元素删除
}
sink {
  StarRocks {
    data_save_mode="APPEND_DATA"
    plugin_input="Table9210050164000"
    nodeUrls = ["10.96.9.163:8040"]
    base-url = "jdbc:mysql://10.96.9.163:9030/"
    username = root
    password = "Hdic@2024"
    database = "market_new_dw"
    table = "t_electronic_back_order_st"
    batch_max_rows = 50000
    starrocks.config = {
      format = "JSON"
      strip_outer_array = true
    }
  }
}

Running Command

本地运行seatunnelapiengine

Error Exception

2025-01-21 14:35:53,104 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:2, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 312b8ad2f737dc000dab053c4c31ec88, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [8d526581a32eee53228efec3b7448325].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [715fa16a7658db0433d1ed1ec27720d7].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [238f4b06c404f694e53eb9f904890a91].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [9769e4a270e5cb4d032284a61fc66a69].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [e16bfd6441bb46f39f147b2323cbaf73].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [d8bc735fcd1ffb89714798b4487e8591].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [09cdbcd9b31c8845bccf8d0b83314b53].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [ebe49ac381a1e17b23eedee8467703fc].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [8a52d914b61e70d970410f3251ecabad].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [d5386e309d3bdf33c942e5deb7f9aeb1].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:0, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 8d526581a32eee53228efec3b7448325, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [4f0a25d58fe7c292ac61888b75a3f116].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [0843be047f8b518352c1f807d3333543].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [066aa4b73d5b5c2fa7c38b8181c11db4].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:8, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 715fa16a7658db0433d1ed1ec27720d7, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [0d11388c59eb70fa77b7dcc0d8eae32f].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool - Releasing slot [d1a8c8c1fa7a55477c20a7ddb82ddf9b].
2025-01-21 14:35:53,105 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:15, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 238f4b06c404f694e53eb9f904890a91, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:11, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 9769e4a270e5cb4d032284a61fc66a69, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:14, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: e16bfd6441bb46f39f147b2323cbaf73, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:1, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: d8bc735fcd1ffb89714798b4487e8591, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:6, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 09cdbcd9b31c8845bccf8d0b83314b53, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:4, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: ebe49ac381a1e17b23eedee8467703fc, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:10, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 8a52d914b61e70d970410f3251ecabad, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:9, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: d5386e309d3bdf33c942e5deb7f9aeb1, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,106 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:7, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 4f0a25d58fe7c292ac61888b75a3f116, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shutting down rest endpoint.
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:12, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 0843be047f8b518352c1f807d3333543, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.jobmaster.JobMaster - Stopping the JobMaster for job 'SeaTunnel_market_new_58_163_sr_Job' (3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:5, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 066aa4b73d5b5c2fa7c38b8181c11db4, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:13, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: 0d11388c59eb70fa77b7dcc0d8eae32f, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,107 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot TaskSlot(index:3, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=64.000gb (68719476736 bytes), taskOffHeapMemory=64.000gb (68719476736 bytes), managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb (4194304 bytes)}, allocationId: d1a8c8c1fa7a55477c20a7ddb82ddf9b, jobId: 3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,110 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator - Closing SourceCoordinator for source Source: Jdbc-Source.
2025-01-21 14:35:53,111 INFO  org.apache.seatunnel.api.event.LoggingEventHandler - log event: EnumeratorCloseEvent(createdTime=1737441353111, jobId=3ae0d2854e7cd5a71ec3bbcf11f5bcf9, eventType=LIFECYCLE_ENUMERATOR_CLOSE)
2025-01-21 14:35:53,119 INFO  org.apache.flink.runtime.source.coordinator.SourceCoordinator - Source coordinator for source Source: Jdbc-Source closed.
2025-01-21 14:35:53,111 INFO  org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService - Stop job leader service.
2025-01-21 14:35:53,119 INFO  org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager - Shutting down TaskExecutorLocalStateStoresManager.
2025-01-21 14:35:53,139 INFO  org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down
2025-01-21 14:35:53,139 INFO  org.apache.flink.runtime.jobmaster.JobMaster - Close ResourceManager connection 67ce06bd98dd66be20349bf0c3a9cea5: Stopping JobMaster for job 'SeaTunnel_market_new_58_163_sr_Job' (3ae0d2854e7cd5a71ec3bbcf11f5bcf9).
2025-01-21 14:35:53,141 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Removing cache directory C:\Users\YL\AppData\Local\Temp\flink-web-ui
2025-01-21 14:35:53,141 INFO  org.apache.flink.runtime.io.disk.FileChannelManagerImpl - FileChannelManager removed spill file directory C:\Users\YL\AppData\Local\Temp\flink-io-e0965170-6783-4367-a3c7-797356f17d3e
2025-01-21 14:35:53,141 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Disconnect job manager 8b53592c77249f7697a9e71202b44f7c@akka://flink/user/rpc/jobmanager_3 for job 3ae0d2854e7cd5a71ec3bbcf11f5bcf9 from the resource manager.
2025-01-21 14:35:53,141 INFO  org.apache.flink.runtime.io.network.NettyShuffleEnvironment - Shutting down the network environment and its components.
2025-01-21 14:35:53,142 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shut down complete.
2025-01-21 14:35:53,142 INFO  org.apache.flink.runtime.io.disk.FileChannelManagerImpl - FileChannelManager removed spill file directory C:\Users\YL\AppData\Local\Temp\flink-netty-shuffle-b192132f-0884-4841-86db-344bac8f79ed
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.taskexecutor.KvStateService - Shutting down the kvState service and its components.
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService - Stop job leader service.
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.filecache.FileCache - removed file cache directory C:\Users\YL\AppData\Local\Temp\flink-dist-cache-47771e53-5fa4-4dc4-876c-eea87310af26
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopped TaskExecutor akka://flink/user/rpc/taskmanager_0.
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Shut down cluster because application is in CANCELED, diagnostics DispatcherResourceManagerComponent has been closed..
2025-01-21 14:35:53,143 INFO  org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent - Closing components.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess - Stopping SessionDispatcherLeaderProcess.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping dispatcher akka://flink/user/rpc/dispatcher_2.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl - Stopping resource manager service.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping all currently running jobs of dispatcher akka://flink/user/rpc/dispatcher_2.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager - Closing the slot manager.
2025-01-21 14:35:53,144 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager - Suspending the slot manager.
2025-01-21 14:35:53,144 INFO  org.apache.seatunnel.core.starter.flink.execution.FlinkExecution - Job finished, execution result: 

***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2025-01-21 14:35:35
End Time                  : 2025-01-21 14:35:53
Total Time(s)             :                  17
Total Read Count          :                   0
Total Write Count         :                   0
Total Read Bytes          :                   0
Total Write Bytes         :                   0
***********************************************

Zeta or Flink or Spark Version

无

Java or Scala Version

11

Screenshots

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[andyuan1000]

[andyuan1000]

11