[ Award ] Stability Problem Collection

[luzhijing]

Hi the community,

In order to further improve the stability of Apache Doris , we will collect the stability issues for the whole community from now on.
If you meet some problems such as Frontend & Backend Crash , Backend Abnormal OOM etc.
Please Contact Us , We will help you solve the problems , and if it is confirmed as a BUG , we will fix it in the subsequent versions as soon as possible .

You can also provide the following information so that we can locate the fault causes faster:

• Doris vision: e.g. 0.14.0 Release
• Error message:
• What operation cause the problem: Query , Load , ect.
• Performance:
• OS: e.g. Centos , Ubantu or else
• Machine configuration: CPU Core , Memory , Storage
• Architecture: e.g. x86 or ARM
• Whether it can be Repetition or not and Repetition Step:
• Relevant Issuees: #XXXXX

——————————

为了进一步提升 Apache Doris 的稳定性，从2021年Q4起我们将会持续向社区征集稳定性问题。

如果您遇到了诸如 FE&BE挂掉 或 BE异常OOM 之类的问题，请及时与我们取得联系，我们将会帮您解决问题。如果确认是Bug的话，我们将会尽快修复并在后续版本中完成上线，并且有Apache Doris周边礼品送出～

为了便于可以更好定位问题，可以提供以下信息：

• 系统版本：例如 0.14.0 Release
• 报错信息：
• 执行了什么操作导致的报错：比如执行的SQL、导入任务的数据量及频率等
• 具体表现：
• 操作系统：
• 机器配置：CPU核数、内存、磁盘
• 平台：x86 或 ARM平台
• 是否能复现以及复现步骤
• 相关Issue：

[lollipopcs]

• 系统版本：0.15.0-rc03

• 报错信息：
  建表语句中使用了String类型字段。
  1.在运行flink程序开窗后，通过stream load实时写入数据到doris；
  2.使用insert into select 批量导入数据到doris。
  doris的各个日志没有明确报错信息，be节点会无故崩溃挂掉。

• 具体表现：
  fe节点正常，但是be节点会无故崩溃挂掉。

• 操作系统：
  centOS 7

• 机器配置：
  测试环境：3台be节点，8核CPU，16G内存，500G HDD。其中一台 BE 混部一个 FE 节点，无Observer。
  生产环境：3台be节点，8核CPU，32G内存，200G SSD。1台fe节点，8核CPU，16G内存，100G SSD。其中一台 BE 混部一个 Observer FE。

• 平台：x86

• 是否能复现以及复现步骤：
  通过以上操作可以复现。

• 相关Issue：
  同步业务库到doris，做宽表提供报表查询使用。
  业务库数据量不大，测试环境每张表只有500W条或更少数据。

问题1：
String类型的字段用在where过滤条件，执行相应sql，导致be节点崩溃挂掉。

问题2：
重建表，并避免所有会涉及到where过滤条件的字段使用String类型，改用varchar。只在类似于memo这种字段使用String，做明细查询使用。
测试环境doris集群，增量数据不多，一开始运行正常，但有时be会偶尔挂掉，起初通过 dmesg -T 查看到系统有kill掉be，一开始会以为是测试环境配置较低，内存不够导致。
随后flink端有大量增量数据存入doris，或者使用insert into select批量导入数据到doris，be节点会无故崩溃挂掉。
查看doris的各个日志没有明确报错信息，通过 dmesg -T查看，内存够用，系统也没有kill掉be。
be.INFO:

be.out日志会有相关问题信息：

随后安装使用 Prometheus 和 Grafana 进项监控，发现be men 内存使用到3G左右，be就会挂掉（不一定是3G）：

为了确定不是资源问题，使用生产环境机器做以上操作，生产环境的be同样会奔溃挂掉。
从而确定不是资源问题导致的。

最后猜测可能还是String类型导致的问题，随后重新建表，并去除所有String类型，全部改用varchar。然后持续导入数据进行压测，doris集群稳定运行。

从而确定是String类型导致doris集群不稳定与be节点的崩溃。

[yangzhg]

0.15.1-rc09 may fix it, you can try it

[guonc]

系统版本：PALO分支的0.14.13/0.14.13.1/0.15.1-rc08

报错信息：
经单机、集群、不同硬件、是否自己编译等多次测试。结论：fe无报错信息，be报错信息如下
start time: Wed Oct 20 15:44:31 CST 2021
*** Aborted at 1634723518 (unix time) try "date -d @1634723518" if you are using GNU date ***
PC: @ 0x113941a doris::SlotRef::get_string_val()
*** SIGSEGV (@0x0) received by PID 26168 (TID 0x7fc03b949700) from PID 0; stack trace: ***
@ 0x1fd12e2 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7fc0f8d6a630 (unknown)
@ 0x113941a doris::SlotRef::get_string_val()
@ 0x1122df8 doris::ExprContext::get_value()
@ 0x11336a7 doris::ScalarFnCall::evaluate_children()
@ 0x11376ae doris::ScalarFnCall::interpret_eval<>()
@ 0x119c0c3 doris::EqDoubleValPred::get_boolean_val()
@ 0x18d7d82 doris::ExecNode::eval_conjuncts()
@ 0x1a3d191 doris::BaseScanner::fill_dest_tuple()
@ 0x1a2b84c doris::JsonScanner::get_next()
@ 0x19f1681 doris::BrokerScanNode::scanner_scan()
@ 0x19f21cc doris::BrokerScanNode::scanner_worker()
@ 0x2ff66e0 execute_native_thread_routine
@ 0x7fc0f8d62ea5 start_thread
@ 0x7fc0f87899fd __clone

具体表现：
Doris创建表，使用Routine Load 前置过滤 PRECEDING FILTER n_region_id = 1 and s_alias = 'XXXX' 从Kafka接收数据。
发现 使用Routine Load 功能会发生不定期be节点crash的问题（多到一天3次以上，可以做保活，但是这样解决不了根本问题）。
而后偶然发现使用 WHERE n_region_id = 1 and s_alias = 'XXXX' 就没有此问题。

操作系统：
centOS 7

机器配置：
3台be节点，8核CPU，32G内存，4T SSD。1台fe节点，8核CPU，16G内存，1T SSD。同时使用be三个节点做fe高可用（2Fl 1Ob）。

平台：x86

是否能复现以及复现步骤：
建表语句：

CREATE TABLE IF NOT EXISTS gc_test (
    n_roleid BIGINT(20) NOT NULL COMMENT '',
    d_create DATETIME NOT NULL COMMENT '',
    xx BIGINT(20) NOT NULL COMMENT '',
    xx VARCHAR(32) NOT NULL COMMENT '',
    xx INT(11) NOT NULL COMMENT '',
    n_region_id INT(11) NOT NULL COMMENT '',
    s_alias VARCHAR(80) NOT NULL COMMENT ''
    )
    UNIQUE KEY(n_roleid,d_create)
    PARTITION BY RANGE(d_create) ()
    DISTRIBUTED BY HASH(n_roleid) BUCKETS 8
    PROPERTIES (
                   "dynamic_partition.create_history_partition" ="true" ,
                   "dynamic_partition.enable" = "true",
                   "dynamic_partition.time_unit" = "MONTH",
                   "dynamic_partition.start" = "-3",
                   "dynamic_partition.end" = "2",
                   "dynamic_partition.prefix" = "p",
                   "dynamic_partition.buckets" = "8",
                   "dynamic_partition.start_day_of_month" = "1",

                   "replication_num" = "3"
               );

创建 Routine Load：

CREATE ROUTINE LOAD test_db.gc_test_routine_load ON gc_test
COLUMNS(
n_roleid,
d_create,
xx,
xx,
xx,
n_region_id,
s_alias
),
PRECEDING FILTER n_region_id = 1 and s_alias = 'XXXX'
PROPERTIES
(
    "desired_concurrent_number"="1",
    "max_batch_interval" = "20",
    "max_batch_rows" = "300000",
    "max_batch_size" = "209715200",
    "strict_mode" = "false",
    "format" = "json"
)
FROM KAFKA
(
    "kafka_broker_list" = "xx",
    "kafka_topic" = "xx",
    "property.group.id" = "xx",
    "property.client.id" = "xx",
    "property.kafka_default_offsets" = "OFFSET_BEGINNING"
);

Kafka 中消息：
每5S最多2000条，有重复
{"n_region_id":"1","s_alias":"XXXX",....}

最后解决办法：
PRECEDING FILTER 换成 WHERE，就可以规避此问题。

[morningman]

Fixed in #7193, will be release in next version, or you can patch it and compile by yourself.

[fvhan]

Doris vision: 0.14.13.1 & 0.15.1-rc09
Performance:
OS: Centos 7
Machine configuration: CPU 40 Core , Memory 64G
Architecture: x86
Error message:
###############################################################################################################
be crash时 be.out文件的日志:
*** Aborted at 1637564200 (unix time) try "date -d @1637564200" if you are using GNU date ***
PC: @ 0x13580d6 doris::RowDescriptor::tuple_is_nullable()
*** SIGSEGV (@0x0) received by PID 3190926 (TID 0x7fff665de700) from PID 0; stack trace: ***
@ 0x1ec6742 google::(anonymous namespace)::FailureSignalHandler()
@ 0x7ffff77c25d0 (unknown)
@ 0x13580d6 doris::RowDescriptor::tuple_is_nullable()
@ 0x1122027 doris::TupleIsNullPredicate::prepare()
@ 0x10e93b2 doris::Expr::prepare()
@ 0x10f3a33 doris::ExprContext::prepare()
@ 0x10ea2de doris::Expr::prepare()
@ 0x1a9a1e3 doris::ResultSink::prepare_exprs()
@ 0x1a9a9b7 doris::ResultSink::prepare()
@ 0x142912b doris::PlanFragmentExecutor::prepare()
@ 0x13a2b4e doris::FragmentExecState::prepare()
@ 0x13a6a27 doris::FragmentMgr::exec_plan_fragment()
@ 0x13a866d doris::FragmentMgr::exec_plan_fragment()
@ 0x146db24 doris::PInternalServiceImpl<>::_exec_plan_fragment()
@ 0x146dbdf doris::PInternalServiceImpl<>::exec_plan_fragment()
@ 0x2039477 brpc::policy::ProcessHttpRequest()
@ 0x2006897 brpc::ProcessInputMessage()
@ 0x200775e brpc::InputMessenger::OnNewMessages()
@ 0x20afd0e brpc::Socket::ProcessEvent()
@ 0x215864f bthread::TaskGroup::task_runner()
@ 0x2149a51 bthread_make_fcontext

################################################################################################################
be crash时 core dump文件用gdb分析的结果：
(gdb) bt
#0 0x00000000013580d6 in operator bool (this=) at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_bvector.h:916
#1 operator* (this=) at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_bvector.h:348
#2 operator[] (__n=, this=) at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_bvector.h:918
#3 doris::RowDescriptor::tuple_is_nullable (this=0x64371b1f8, tuple_idx=-1398457043) at ../src/runtime/descriptors.cpp:357
#4 0x0000000001122027 in doris::TupleIsNullPredicate::prepare(doris::RuntimeState*, doris::RowDescriptor const&, doris::ExprContext*) ()
at ../src/exprs/tuple_is_null_predicate.cpp:39
#5 0x00000000010e93b2 in doris::Expr::prepare(doris::RuntimeState*, doris::RowDescriptor const&, doris::ExprContext*) ()
at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_vector.h:1043
#6 0x00000000010f3a33 in doris::ExprContext::prepare(doris::RuntimeState*, doris::RowDescriptor const&, std::shared_ptrdoris::MemTracker const&) ()
at ../src/exprs/expr_context.cpp:61
#7 0x00000000010ea2de in doris::Expr::prepare(std::vector<doris::ExprContext*, std::allocatordoris::ExprContext* > const&, doris::RuntimeState*, doris::RowDescriptor const&, std::shared_ptrdoris::MemTracker const&) () at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_vector.h:1061
#8 0x0000000001a9a1e3 in doris::ResultSink::prepare_exprs(doris::RuntimeState*) () at ../src/runtime/result_sink.cpp:57
#9 0x0000000001a9a9b7 in doris::ResultSink::prepare(doris::RuntimeState*) () at ../src/runtime/result_sink.cpp:69
#10 0x000000000142912b in doris::PlanFragmentExecutor::prepare(doris::TExecPlanFragmentParams const&, doris::QueryFragmentsCtx const*) ()
at /var/local/thirdparty/installed/include/boost/smart_ptr/scoped_ptr.hpp:109
#11 0x00000000013a2b4e in doris::FragmentExecState::prepare (this=this@entry=0x159272500, params=...) at ../src/runtime/fragment_mgr.cpp:229
#12 0x00000000013a6a27 in doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::PlanFragmentExecutor*)>)
() at ../src/runtime/fragment_mgr.cpp:609
#13 0x00000000013a866d in doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&) ()
at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/std_function.h:87
#14 0x000000000146db24 in doris::PInternalServiceImpldoris::PBackendService::_exec_plan_fragment (this=this@entry=0xa87bb80, ser_request=...)
at ../src/runtime/exec_env.h:117
---Type to continue, or q to quit---
#15 0x000000000146dbdf in doris::PInternalServiceImpldoris::PBackendService::exec_plan_fragment (this=0xa87bb80, cntl_base=,
request=, response=0xe9e1bdc0, done=0x1721923c0) at /var/local/thirdparty/installed/include/google/protobuf/arenastring.h:231
#16 0x0000000002039477 in brpc::policy::ProcessHttpRequest(brpc::InputMessageBase*) () at ../src/brpc/policy/http_rpc_protocol.cpp:1484
#17 0x0000000002006897 in brpc::ProcessInputMessage (void_arg=void_arg@entry=0x12f5ac6d0) at ../src/brpc/input_messenger.cpp:135
#18 0x000000000200775e in operator() (last_msg=0x12f5ac6d0, this=) at ../src/brpc/input_messenger.cpp:141
#19 ~unique_ptr (this=, __in_chrg=) at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/unique_ptr.h:361
#20 brpc::InputMessenger::OnNewMessages(brpc::Socket*) () at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/unique_ptr.h:355
#21 0x00000000020afd0e in brpc::Socket::ProcessEvent(void*) () at ../src/brpc/socket.cpp:1017
#22 0x000000000215864f in bthread::TaskGroup::task_runner(long) () at ../src/bthread/task_group.cpp:296
#23 0x0000000002149a51 in bthread_make_fcontext () at /opt/rh/devtoolset-10/root/usr/include/c++/10/bits/stl_iterator.h:953
Cannot access memory at address 0x7fff3cb75000
(gdb) f 10
#10 0x000000000142912b in doris::PlanFragmentExecutor::prepare(doris::TExecPlanFragmentParams const&, doris::QueryFragmentsCtx const*) ()
at /var/local/thirdparty/installed/include/boost/smart_ptr/scoped_ptr.hpp:109
109 /var/local/thirdparty/installed/include/boost/smart_ptr/scoped_ptr.hpp: No such file or directory.
(gdb) p _query_id
$1 = {_vptr.TUniqueId = 0x2f9e558 <vtable for doris::TUniqueId+16>, hi = -2936670627391388654, lo = -6006327264025392940}
(gdb) p /x -2936670627391388654
$2 = 0xd73ed99fa0034412
##########################################################################################################
select * from ( SELECT * FROM dwd_telemarketing_cust_market_register_dtl_v) user_defined_view limit 0 #be节点会崩溃
这个视图查询语句会导致be节点崩溃,fe节点正常，未在be节点的日志中发现明显异常
我们验证的结果是：
不加limit 0 就不会导致be节点崩溃：
SELECT * FROM dwd_telemarketing_cust_market_register_dtl_vlimit 0 ;#会崩溃
SELECT * FROM dwd_telemarketing_cust_market_register_dtl_v limit 1;#不会导致be崩溃
SELECT * FROM dwd_telemarketing_cust_market_register_dtl_vlimit 100000;#不会导致be崩溃
SELECT * FROM dwd_telemarketing_cust_market_register_dtl_v;#不会导致be崩溃
相关建表语句和视图语句分别为：

###################################################建表语句###############################################

CREATE TABLE `model_smt_mdm_user` (
  `id` int(11) NOT NULL ,
  `email` varchar(64) NULL ,
  `icc_account` varchar(64) NULL ,
  `create_time` datetime NULL ,
  `name` varchar(32) NULL ,
  `mobile` varchar(16) NULL ,
  `empno` varchar(32) NULL ,
  `emp_password` varchar(64) NULL ,
  `extension_no` int(11) NULL ,
  `extension_password` varchar(64) NULL ,
  `org_id` int(11) NULL ,
  `org_name` varchar(256) NULL ,
  `position_id` int(11) NULL ,
  `position_name` varchar(64) NULL ,
  `assign_limit_day` int(11) NULL ,
  `assign_limit_once` int(11) NULL ,
  `status` tinyint(4) NULL ,
  `create_userid` int(11) NULL ,
  `create_user_name` varchar(64) NULL ,
  `update_time` datetime NULL ,
  `update_userid` int(11) NULL ,
  `update_user_name` varchar(64) NULL ,
  `is_delete` tinyint(4) NULL ,
  `is_examine` tinyint(4) NULL DEFAULT "0" ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(32) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `email`, `icc_account`, `create_time`)
DISTRIBUTED BY HASH(`id`, `email`, `icc_account`, `create_time`) BUCKETS 1
PROPERTIES (
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_smt_zib_call_result` (
  `id` int(11) NOT NULL ,
  `call_recordid` int(11) NULL ,
  `create_time` datetime NULL ,
  `market_order_id` bigint(20) NULL ,
  `activity_customerid` int(11) NULL ,
  `activity_id` varchar(32) NULL ,
  `activity_name` varchar(128) NULL DEFAULT "" ,
  `batch_no` varchar(255) NULL DEFAULT "" ,
  `customer_id` varchar(64) NULL ,
  `mobile` varchar(16) NULL ,
  `customer_name` varchar(32) NULL ,
  `action_first_code` varchar(32) NULL ,
  `action_second_code` varchar(32) NULL ,
  `status` tinyint(4) NULL ,
  `order_status` tinyint(4) NULL DEFAULT "0" ,
  `is_invite` tinyint(4) NULL ,
  `remark` varchar(512) NULL ,
  `user_id` int(11) NULL ,
  `user_name` varchar(32) NULL ,
  `user_empno` varchar(32) NULL ,
  `vgroup_id` int(11) NULL ,
  `vgroup_name` varchar(32) NULL ,
  `org_id` int(11) NULL ,
  `org_name` varchar(32) NULL ,
  `create_userid` int(11) NULL ,
  `update_time` datetime NULL ,
  `update_userid` int(11) NULL ,
  `is_delete` tinyint(4) NULL ,
  `type` tinyint(4) NULL ,
  `ignored` tinyint(4) NULL DEFAULT "0" ,
  `level` tinyint(4) NULL ,
  `first_dial_time` datetime NULL ,
  `last_dial_time` datetime NULL ,
  `success_count` int(11) NULL ,
  `today_dial_count` int(11) NULL DEFAULT "0" ,
  `cruise_call_result` varchar(256) NULL DEFAULT "" ,
  `mobile_enc` varchar(128) NULL ,
  `next_visit_time` varchar(10) NULL DEFAULT "" ,
  `valid_score` int(11) NULL DEFAULT "0" ,
  `customer_name_enc` varchar(128) NULL ,
  `book_time` varchar(20) NULL DEFAULT "" ,
  `last_visit_time` datetime NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(32) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `call_recordid`, `create_time`)
DISTRIBUTED BY HASH(`id`, `call_recordid`, `create_time`) BUCKETS 4
PROPERTIES (
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_resu_info` (
  `id` bigint(20) NOT NULL ,
  `time_inst` datetime NOT NULL ,
  `ident` varchar(20) NULL ,
  `mobile` varchar(64) NOT NULL ,
  `unique_id` varchar(32) NULL ,
  `cust_id` bigint(20) NULL ,
  `name` varchar(60) NULL ,
  `name_enc` varchar(240) NULL ,
  `ident_enc` varchar(80) NULL ,
  `mobile_enc` varchar(80) NULL ,
  `password` varchar(50) NOT NULL ,
  `reg_time` datetime NOT NULL ,
  `modify_time` varchar(30) NOT NULL ,
  `reg_source` varchar(20) NOT NULL ,
  `reg_app_channel` varchar(20) NULL ,
  `reg_device_info` varchar(500) NULL ,
  `status` varchar(20) NULL ,
  `is_mobile_authed` char(1) NOT NULL DEFAULT "0" ,
  `complate_cust_info` varchar(3) NULL ,
  `time_upd` datetime NOT NULL ,
  `has_checked` varchar(3) NULL DEFAULT "0" ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(64) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `time_inst`)
PARTITION BY RANGE(`time_inst`)
(PARTITION p1 VALUES [('0000-01-01 00:00:00'), ('2016-01-01 00:00:00')),
PARTITION p201601 VALUES [('2016-01-01 00:00:00'), ('2016-02-01 00:00:00')),
PARTITION p201602 VALUES [('2016-02-01 00:00:00'), ('2016-03-01 00:00:00')),
PARTITION p201603 VALUES [('2016-03-01 00:00:00'), ('2016-04-01 00:00:00')),
PARTITION p201604 VALUES [('2016-04-01 00:00:00'), ('2016-05-01 00:00:00')),
PARTITION p201605 VALUES [('2016-05-01 00:00:00'), ('2016-06-01 00:00:00')),
PARTITION p201606 VALUES [('2016-06-01 00:00:00'), ('2016-07-01 00:00:00')),
PARTITION p201607 VALUES [('2016-07-01 00:00:00'), ('2016-08-01 00:00:00')),
PARTITION p201608 VALUES [('2016-08-01 00:00:00'), ('2016-09-01 00:00:00')),
PARTITION p201609 VALUES [('2016-09-01 00:00:00'), ('2016-10-01 00:00:00')),
PARTITION p201610 VALUES [('2016-10-01 00:00:00'), ('2016-11-01 00:00:00')),
PARTITION p201611 VALUES [('2016-11-01 00:00:00'), ('2016-12-01 00:00:00')),
PARTITION p201612 VALUES [('2016-12-01 00:00:00'), ('2017-01-01 00:00:00')),
PARTITION p201701 VALUES [('2017-01-01 00:00:00'), ('2017-02-01 00:00:00')),
PARTITION p201702 VALUES [('2017-02-01 00:00:00'), ('2017-03-01 00:00:00')),
PARTITION p201703 VALUES [('2017-03-01 00:00:00'), ('2017-04-01 00:00:00')),
PARTITION p201704 VALUES [('2017-04-01 00:00:00'), ('2017-05-01 00:00:00')),
PARTITION p201705 VALUES [('2017-05-01 00:00:00'), ('2017-06-01 00:00:00')),
PARTITION p201706 VALUES [('2017-06-01 00:00:00'), ('2017-07-01 00:00:00')),
PARTITION p201707 VALUES [('2017-07-01 00:00:00'), ('2017-08-01 00:00:00')),
PARTITION p201708 VALUES [('2017-08-01 00:00:00'), ('2017-09-01 00:00:00')),
PARTITION p201709 VALUES [('2017-09-01 00:00:00'), ('2017-10-01 00:00:00')),
PARTITION p201710 VALUES [('2017-10-01 00:00:00'), ('2017-11-01 00:00:00')),
PARTITION p201711 VALUES [('2017-11-01 00:00:00'), ('2017-12-01 00:00:00')),
PARTITION p201712 VALUES [('2017-12-01 00:00:00'), ('2018-01-01 00:00:00')),
PARTITION p201801 VALUES [('2018-01-01 00:00:00'), ('2018-02-01 00:00:00')),
PARTITION p201802 VALUES [('2018-02-01 00:00:00'), ('2018-03-01 00:00:00')),
PARTITION p201803 VALUES [('2018-03-01 00:00:00'), ('2018-04-01 00:00:00')),
PARTITION p201804 VALUES [('2018-04-01 00:00:00'), ('2018-05-01 00:00:00')),
PARTITION p201805 VALUES [('2018-05-01 00:00:00'), ('2018-06-01 00:00:00')),
PARTITION p201806 VALUES [('2018-06-01 00:00:00'), ('2018-07-01 00:00:00')),
PARTITION p201807 VALUES [('2018-07-01 00:00:00'), ('2018-08-01 00:00:00')),
PARTITION p201808 VALUES [('2018-08-01 00:00:00'), ('2018-09-01 00:00:00')),
PARTITION p201809 VALUES [('2018-09-01 00:00:00'), ('2018-10-01 00:00:00')),
PARTITION p201810 VALUES [('2018-10-01 00:00:00'), ('2018-11-01 00:00:00')),
PARTITION p201811 VALUES [('2018-11-01 00:00:00'), ('2018-12-01 00:00:00')),
PARTITION p201812 VALUES [('2018-12-01 00:00:00'), ('2019-01-01 00:00:00')),
PARTITION p201901 VALUES [('2019-01-01 00:00:00'), ('2019-02-01 00:00:00')),
PARTITION p201902 VALUES [('2019-02-01 00:00:00'), ('2019-03-01 00:00:00')),
PARTITION p201903 VALUES [('2019-03-01 00:00:00'), ('2019-04-01 00:00:00')),
PARTITION p201904 VALUES [('2019-04-01 00:00:00'), ('2019-05-01 00:00:00')),
PARTITION p201905 VALUES [('2019-05-01 00:00:00'), ('2019-06-01 00:00:00')),
PARTITION p201906 VALUES [('2019-06-01 00:00:00'), ('2019-07-01 00:00:00')),
PARTITION p201907 VALUES [('2019-07-01 00:00:00'), ('2019-08-01 00:00:00')),
PARTITION p201908 VALUES [('2019-08-01 00:00:00'), ('2019-09-01 00:00:00')),
PARTITION p201909 VALUES [('2019-09-01 00:00:00'), ('2019-10-01 00:00:00')),
PARTITION p201910 VALUES [('2019-10-01 00:00:00'), ('2019-11-01 00:00:00')),
PARTITION p201911 VALUES [('2019-11-01 00:00:00'), ('2019-12-01 00:00:00')),
PARTITION p201912 VALUES [('2019-12-01 00:00:00'), ('2020-01-01 00:00:00')),
PARTITION p202001 VALUES [('2020-01-01 00:00:00'), ('2020-02-01 00:00:00')),
PARTITION p202002 VALUES [('2020-02-01 00:00:00'), ('2020-03-01 00:00:00')),
PARTITION p202003 VALUES [('2020-03-01 00:00:00'), ('2020-04-01 00:00:00')),
PARTITION p202004 VALUES [('2020-04-01 00:00:00'), ('2020-05-01 00:00:00')),
PARTITION p202005 VALUES [('2020-05-01 00:00:00'), ('2020-06-01 00:00:00')),
PARTITION p202006 VALUES [('2020-06-01 00:00:00'), ('2020-07-01 00:00:00')),
PARTITION p202007 VALUES [('2020-07-01 00:00:00'), ('2020-08-01 00:00:00')),
PARTITION p202008 VALUES [('2020-08-01 00:00:00'), ('2020-09-01 00:00:00')),
PARTITION p202009 VALUES [('2020-09-01 00:00:00'), ('2020-10-01 00:00:00')),
PARTITION p202010 VALUES [('2020-10-01 00:00:00'), ('2020-11-01 00:00:00')),
PARTITION p202011 VALUES [('2020-11-01 00:00:00'), ('2020-12-01 00:00:00')),
PARTITION p202012 VALUES [('2020-12-01 00:00:00'), ('2021-01-01 00:00:00')),
PARTITION p202101 VALUES [('2021-01-01 00:00:00'), ('2021-02-01 00:00:00')),
PARTITION p202102 VALUES [('2021-02-01 00:00:00'), ('2021-03-01 00:00:00')),
PARTITION p202103 VALUES [('2021-03-01 00:00:00'), ('2021-04-01 00:00:00')),
PARTITION p202104 VALUES [('2021-04-01 00:00:00'), ('2021-05-01 00:00:00')),
PARTITION p202105 VALUES [('2021-05-01 00:00:00'), ('2021-06-01 00:00:00')),
PARTITION p202106 VALUES [('2021-06-01 00:00:00'), ('2021-07-01 00:00:00')),
PARTITION p202107 VALUES [('2021-07-01 00:00:00'), ('2021-08-01 00:00:00')),
PARTITION p202108 VALUES [('2021-08-01 00:00:00'), ('2021-09-01 00:00:00')),
PARTITION p202109 VALUES [('2021-09-01 00:00:00'), ('2021-10-01 00:00:00')),
PARTITION p202110 VALUES [('2021-10-01 00:00:00'), ('2021-11-01 00:00:00')),
PARTITION p202111 VALUES [('2021-11-01 00:00:00'), ('2021-12-01 00:00:00')),
PARTITION p202112 VALUES [('2021-12-01 00:00:00'), ('2022-01-01 00:00:00')))
DISTRIBUTED BY HASH(`time_inst`) BUCKETS 1
PROPERTIES (
"replication_num" = "3",
"dynamic_partition.enable" = "false",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2147483648",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_num" = "3",
"dynamic_partition.buckets" = "1",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.start_day_of_month" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_resu_info_back` (
  `id` bigint(20) NOT NULL ,
  `time_inst` datetime NOT NULL ,
  `unique_id` varchar(32) NULL ,
  `ident` varchar(20) NULL ,
  `cust_id` bigint(20) NULL ,
  `name` varchar(60) NULL ,
  `name_enc` varchar(240) NULL ,
  `ident_enc` varchar(80) NULL ,
  `mobile` varchar(64) NOT NULL ,
  `mobile_enc` varchar(80) NULL ,
  `password` varchar(50) NOT NULL ,
  `reg_time` datetime NOT NULL ,
  `modify_time` varchar(30) NOT NULL ,
  `reg_source` varchar(20) NOT NULL ,
  `reg_app_channel` varchar(20) NULL ,
  `reg_device_info` varchar(500) NULL ,
  `status` varchar(20) NULL ,
  `is_mobile_authed` char(1) NOT NULL DEFAULT "0" ,
  `complate_cust_info` varchar(3) NULL ,
  `time_upd` datetime NOT NULL ,
  `has_checked` varchar(3) NULL DEFAULT "0" ,
  `oper` varchar(50) NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(64) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `time_inst`)
DISTRIBUTED BY HASH(`time_inst`) BUCKETS 6
PROPERTIES (
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_yllpa_main` (
  `apply_no` varchar(32) NOT NULL ,
  `unique_id` varchar(32) NOT NULL ,
  `apply_time` datetime NOT NULL ,
  `apply_serial_no` varchar(64) NULL ,
  `product_code` varchar(4) NULL ,
  `cooperator_id` varchar(8) NULL ,
  `cooperator_name` varchar(45) NULL ,
  `asset_type` varchar(16) NULL ,
  `pre_apply_no` varchar(32) NULL ,
  `status` varchar(32) NULL ,
  `rtf_state` varchar(32) NULL ,
  `refuse_code` varchar(200) NULL ,
  `refuse_desc` varchar(500) NULL ,
  `contract_no` varchar(32) NULL ,
  `sub_terminal_type` varchar(50) NULL ,
  `apply_type` varchar(10) NULL ,
  `time_inst` datetime NULL ,
  `time_upd` datetime NULL ,
  `comp_time` datetime NULL ,
  `acc_lmt` decimal(19, 4) NULL ,
  `order_id` varchar(32) NULL ,
  `workflow_flag` varchar(10) NULL ,
  `apply_lmt` decimal(19, 4) NULL ,
  `app_loan_term` int(11) NULL ,
  `loan_term` int(11) NULL ,
  `sug_lmt` decimal(19, 4) NULL ,
  `loan_purpose` varchar(10) NULL ,
  `join_life_insurance` char(1) NULL ,
  `loan_fixed_amt` decimal(19, 4) NULL ,
  `due_bill_no` varchar(32) NULL ,
  `apply_from_type` varchar(10) NULL ,
  `memo` varchar(500) NULL ,
  `app_remark` varchar(500) NULL ,
  `confirm_date` datetime NULL ,
  `sub_product_code` varchar(20) NULL ,
  `loan_rate` decimal(11, 8) NULL ,
  `execute_rate` decimal(10, 8) NULL ,
  `fee_rate` decimal(11, 8) NULL ,
  `fee_amt` decimal(19, 4) NULL ,
  `penalty_rate` decimal(10, 8) NULL ,
  `life_insu_fee_rate` decimal(10, 8) NULL ,
  `life_insu_amt` decimal(19, 4) NULL ,
  `installment_fee_rate` decimal(10, 8) NULL ,
  `installment_fee_amt` decimal(19, 4) NULL ,
  `cash_limit_rate` decimal(10, 8) NULL ,
  `pocket_money_limit_rate` decimal(10, 8) NULL ,
  `cash_handling_fee_rate` decimal(10, 8) NULL ,
  `cash_handling_fee_amount` decimal(19, 4) NULL ,
  `ins_rate` decimal(10, 8) NULL ,
  `ins_amt` decimal(19, 4) NULL ,
  `agent_fee_rate` decimal(10, 8) NULL ,
  `agent_fee_amt` decimal(19, 4) NULL ,
  `guaranty_id` varchar(32) NULL ,
  `central_bank_credit_auth_flag` varchar(1) NULL ,
  `social_auth_flag` varchar(1) NULL ,
  `providers_auth_flag` varchar(1) NULL ,
  `sesame_auth_flag` varchar(1) NULL ,
  `auth_flag_ext` varchar(100) NULL ,
  `credit_auth_serial_no` varchar(50) NULL ,
  `patch_type` varchar(50) NULL ,
  `patch_type_complete` varchar(50) NULL ,
  `patch_num` int(11) NULL ,
  `patch_cause` varchar(1024) NULL ,
  `data_complete_type` varchar(256) NULL ,
  `prepay_pkg_ind` varchar(10) NULL ,
  `is_old_customer` varchar(10) NULL ,
  `creadit_report_id` varchar(64) NULL ,
  `market_product_id` varchar(32) NULL ,
  `canvass_approve_state` varchar(32) NULL ,
  `premium_ind` varchar(1) NULL ,
  `premium_Amt` decimal(19, 2) NULL ,
  `client_contract_sign_time` datetime NULL ,
  `fail_reason` varchar(400) NULL ,
  `contract_exp_time` datetime NULL ,
  `apply_channel` varchar(10) NULL ,
  `prepay_pkg_ind_amt` decimal(19, 4) NULL ,
  `prepay_pkg_ind_rate` decimal(10, 8) NULL ,
  `join_screen_insu_ind` char(1) NULL ,
  `screen_insu_fee_rate` decimal(10, 8) NULL ,
  `screen_insu_fee_amt` decimal(19, 4) NULL ,
  `acct_type` varchar(16) NULL ,
  `credit_business_flag` varchar(32) NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(32) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`apply_no`, `unique_id`, `apply_time`)
PARTITION BY RANGE(`apply_time`)
(PARTITION p201511 VALUES [('0000-01-01 00:00:00'), ('2015-12-01 00:00:00')),
PARTITION p201512 VALUES [('2015-12-01 00:00:00'), ('2016-01-01 00:00:00')),
PARTITION p201601 VALUES [('2016-01-01 00:00:00'), ('2016-02-01 00:00:00')),
PARTITION p201602 VALUES [('2016-02-01 00:00:00'), ('2016-03-01 00:00:00')),
PARTITION p201603 VALUES [('2016-03-01 00:00:00'), ('2016-04-01 00:00:00')),
PARTITION p201604 VALUES [('2016-04-01 00:00:00'), ('2016-05-01 00:00:00')),
PARTITION p201605 VALUES [('2016-05-01 00:00:00'), ('2016-06-01 00:00:00')),
PARTITION p201606 VALUES [('2016-06-01 00:00:00'), ('2016-07-01 00:00:00')),
PARTITION p201607 VALUES [('2016-07-01 00:00:00'), ('2016-08-01 00:00:00')),
PARTITION p201608 VALUES [('2016-08-01 00:00:00'), ('2016-09-01 00:00:00')),
PARTITION p201609 VALUES [('2016-09-01 00:00:00'), ('2016-10-01 00:00:00')),
PARTITION p201610 VALUES [('2016-10-01 00:00:00'), ('2016-11-01 00:00:00')),
PARTITION p201611 VALUES [('2016-11-01 00:00:00'), ('2016-12-01 00:00:00')),
PARTITION p201612 VALUES [('2016-12-01 00:00:00'), ('2017-01-01 00:00:00')),
PARTITION p201701 VALUES [('2017-01-01 00:00:00'), ('2017-02-01 00:00:00')),
PARTITION p201702 VALUES [('2017-02-01 00:00:00'), ('2017-03-01 00:00:00')),
PARTITION p201703 VALUES [('2017-03-01 00:00:00'), ('2017-04-01 00:00:00')),
PARTITION p201704 VALUES [('2017-04-01 00:00:00'), ('2017-05-01 00:00:00')),
PARTITION p201705 VALUES [('2017-05-01 00:00:00'), ('2017-06-01 00:00:00')),
PARTITION p201706 VALUES [('2017-06-01 00:00:00'), ('2017-07-01 00:00:00')),
PARTITION p201707 VALUES [('2017-07-01 00:00:00'), ('2017-08-01 00:00:00')),
PARTITION p201708 VALUES [('2017-08-01 00:00:00'), ('2017-09-01 00:00:00')),
PARTITION p201709 VALUES [('2017-09-01 00:00:00'), ('2017-10-01 00:00:00')),
PARTITION p201710 VALUES [('2017-10-01 00:00:00'), ('2017-11-01 00:00:00')),
PARTITION p201711 VALUES [('2017-11-01 00:00:00'), ('2017-12-01 00:00:00')),
PARTITION p201712 VALUES [('2017-12-01 00:00:00'), ('2018-01-01 00:00:00')),
PARTITION p201801 VALUES [('2018-01-01 00:00:00'), ('2018-02-01 00:00:00')),
PARTITION p201802 VALUES [('2018-02-01 00:00:00'), ('2018-03-01 00:00:00')),
PARTITION p201803 VALUES [('2018-03-01 00:00:00'), ('2018-04-01 00:00:00')),
PARTITION p201804 VALUES [('2018-04-01 00:00:00'), ('2018-05-01 00:00:00')),
PARTITION p201805 VALUES [('2018-05-01 00:00:00'), ('2018-06-01 00:00:00')),
PARTITION p201806 VALUES [('2018-06-01 00:00:00'), ('2018-07-01 00:00:00')),
PARTITION p201807 VALUES [('2018-07-01 00:00:00'), ('2018-08-01 00:00:00')),
PARTITION p201808 VALUES [('2018-08-01 00:00:00'), ('2018-09-01 00:00:00')),
PARTITION p201809 VALUES [('2018-09-01 00:00:00'), ('2018-10-01 00:00:00')),
PARTITION p201810 VALUES [('2018-10-01 00:00:00'), ('2018-11-01 00:00:00')),
PARTITION p201811 VALUES [('2018-11-01 00:00:00'), ('2018-12-01 00:00:00')),
PARTITION p201812 VALUES [('2018-12-01 00:00:00'), ('2019-01-01 00:00:00')),
PARTITION p201901 VALUES [('2019-01-01 00:00:00'), ('2019-02-01 00:00:00')),
PARTITION p201902 VALUES [('2019-02-01 00:00:00'), ('2019-03-01 00:00:00')),
PARTITION p201903 VALUES [('2019-03-01 00:00:00'), ('2019-04-01 00:00:00')),
PARTITION p201904 VALUES [('2019-04-01 00:00:00'), ('2019-05-01 00:00:00')),
PARTITION p201905 VALUES [('2019-05-01 00:00:00'), ('2019-06-01 00:00:00')),
PARTITION p201906 VALUES [('2019-06-01 00:00:00'), ('2019-07-01 00:00:00')),
PARTITION p201907 VALUES [('2019-07-01 00:00:00'), ('2019-08-01 00:00:00')),
PARTITION p201908 VALUES [('2019-08-01 00:00:00'), ('2019-09-01 00:00:00')),
PARTITION p201909 VALUES [('2019-09-01 00:00:00'), ('2019-10-01 00:00:00')),
PARTITION p201910 VALUES [('2019-10-01 00:00:00'), ('2019-11-01 00:00:00')),
PARTITION p201911 VALUES [('2019-11-01 00:00:00'), ('2019-12-01 00:00:00')),
PARTITION p201912 VALUES [('2019-12-01 00:00:00'), ('2020-01-01 00:00:00')),
PARTITION p202001 VALUES [('2020-01-01 00:00:00'), ('2020-02-01 00:00:00')),
PARTITION p202002 VALUES [('2020-02-01 00:00:00'), ('2020-03-01 00:00:00')),
PARTITION p202003 VALUES [('2020-03-01 00:00:00'), ('2020-04-01 00:00:00')),
PARTITION p202004 VALUES [('2020-04-01 00:00:00'), ('2020-05-01 00:00:00')),
PARTITION p202005 VALUES [('2020-05-01 00:00:00'), ('2020-06-01 00:00:00')),
PARTITION p202006 VALUES [('2020-06-01 00:00:00'), ('2020-07-01 00:00:00')),
PARTITION p202007 VALUES [('2020-07-01 00:00:00'), ('2020-08-01 00:00:00')),
PARTITION p202008 VALUES [('2020-08-01 00:00:00'), ('2020-09-01 00:00:00')),
PARTITION p202009 VALUES [('2020-09-01 00:00:00'), ('2020-10-01 00:00:00')),
PARTITION p202010 VALUES [('2020-10-01 00:00:00'), ('2020-11-01 00:00:00')),
PARTITION p202011 VALUES [('2020-11-01 00:00:00'), ('2020-12-01 00:00:00')),
PARTITION p202012 VALUES [('2020-12-01 00:00:00'), ('2021-01-01 00:00:00')),
PARTITION p202101 VALUES [('2021-01-01 00:00:00'), ('2021-02-01 00:00:00')),
PARTITION p202102 VALUES [('2021-02-01 00:00:00'), ('2021-03-01 00:00:00')),
PARTITION p202103 VALUES [('2021-03-01 00:00:00'), ('2021-04-01 00:00:00')),
PARTITION p202104 VALUES [('2021-04-01 00:00:00'), ('2021-05-01 00:00:00')),
PARTITION p202105 VALUES [('2021-05-01 00:00:00'), ('2021-06-01 00:00:00')),
PARTITION p202106 VALUES [('2021-06-01 00:00:00'), ('2021-07-01 00:00:00')),
PARTITION p202107 VALUES [('2021-07-01 00:00:00'), ('2021-08-01 00:00:00')),
PARTITION p202108 VALUES [('2021-08-01 00:00:00'), ('2021-09-01 00:00:00')),
PARTITION p202109 VALUES [('2021-09-01 00:00:00'), ('2021-10-01 00:00:00')),
PARTITION p202110 VALUES [('2021-10-01 00:00:00'), ('2021-11-01 00:00:00')),
PARTITION p202111 VALUES [('2021-11-01 00:00:00'), ('2021-12-01 00:00:00')),
PARTITION p202112 VALUES [('2021-12-01 00:00:00'), ('2022-01-01 00:00:00')),
PARTITION p202201 VALUES [('2022-01-01 00:00:00'), ('2022-02-01 00:00:00')),
PARTITION p202202 VALUES [('2022-02-01 00:00:00'), ('2022-03-01 00:00:00')))
DISTRIBUTED BY HASH(`apply_no`) BUCKETS 4
PROPERTIES (
"replication_num" = "3",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2147483648",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_num" = "3",
"dynamic_partition.buckets" = "4",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.start_day_of_month" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_smt_zib_activity_customer` (
  `id` int(11) NOT NULL ,
  `create_time` datetime NULL ,
  `callid` varchar(32) NULL ,
  `market_order_id` bigint(20) NULL ,
  `order_type` varchar(64) NULL DEFAULT "CALL_NEW" ,
  `batch_no` varchar(255) NULL DEFAULT "" ,
  `activity_id` varchar(32) NULL ,
  `activity_name` varchar(128) NULL DEFAULT "" ,
  `activity_start_time` datetime NULL ,
  `activity_end_time` datetime NULL ,
  `activity_type` varchar(16) NULL ,
  `customer_id` varchar(64) NULL DEFAULT "" ,
  `balance` decimal(10, 2) NULL ,
  `product_id` varchar(32) NULL DEFAULT "" ,
  `customer_name` varchar(64) NULL ,
  `customer_mobile` varchar(16) NULL DEFAULT "" ,
  `invite_code` varchar(64) NULL ,
  `invite_amount` varchar(16) NULL ,
  `invite_id` varchar(32) NULL ,
  `customer_groupid` varchar(32) NULL ,
  `dynamic_batch_no` varchar(64) NULL ,
  `call_start_time` datetime NULL ,
  `call_end_time` datetime NULL ,
  `status` int(11) NULL ,
  `order_status` tinyint(4) NULL DEFAULT "0" ,
  `order_recover_count` int(11) NULL DEFAULT "0" ,
  `biz_id` bigint(20) NULL ,
  `rule_name` varchar(32) NULL DEFAULT "" ,
  `create_userid` int(11) NULL ,
  `update_time` datetime NULL ,
  `update_userid` int(11) NULL ,
  `is_delete` tinyint(4) NULL ,
  `is_missing_customer` tinyint(4) NULL ,
  `commodity_number` int(11) NULL ,
  `is_finance` varchar(8) NULL ,
  `week_time` int(11) NULL ,
  `idle_begin_time` datetime NULL ,
  `idle_end_time` datetime NULL ,
  `month_rate` varchar(32) NULL ,
  `apply_port` varchar(255) NULL ,
  `apply_channel` varchar(32) NULL ,
  `contract_number` varchar(32) NULL ,
  `offer_effective_begin_time` datetime NULL ,
  `offer_effective_end_time` datetime NULL ,
  `call_count` int(11) NULL DEFAULT "0" ,
  `answer_count` int(11) NULL DEFAULT "0" ,
  `business_line` varchar(63) NULL DEFAULT "" ,
  `product_no` varchar(63) NULL DEFAULT "" ,
  `business_category` varchar(63) NULL DEFAULT "" ,
  `label_category` varchar(63) NULL DEFAULT "" ,
  `customer_group` varchar(63) NULL DEFAULT "" ,
  `last_assigned_time` datetime NULL ,
  `customer_mobile_enc` varchar(128) NULL ,
  `customer_name_enc` varchar(128) NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(64) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `create_time`, `callid`, `market_order_id`)
PARTITION BY RANGE(`create_time`)
(PARTITION p202111 VALUES [('2021-10-01 00:00:00'), ('2021-11-01 00:00:00')),
PARTITION p202112 VALUES [('2021-11-01 00:00:00'), ('2021-12-01 00:00:00')),
PARTITION p202201 VALUES [('2022-01-01 00:00:00'), ('2022-02-01 00:00:00')),
PARTITION p202202 VALUES [('2022-02-01 00:00:00'), ('2022-03-01 00:00:00')))
DISTRIBUTED BY HASH(`callid`, `market_order_id`) BUCKETS 8
PROPERTIES (
"replication_num" = "3",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2147483648",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_num" = "3",
"dynamic_partition.buckets" = "8",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.start_day_of_month" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_smt_zib_assign_detail` (
  `id` int(11) NOT NULL ,
  `create_time` datetime NULL ,
  `market_order_id` bigint(20) NULL ,
  `assign_recordid` int(11) NULL DEFAULT "0" ,
  `activity_detailid` int(11) NULL ,
  `vgroup_id` int(11) NULL ,
  `user_id` int(11) NULL ,
  `create_userid` int(11) NULL ,
  `update_time` datetime NULL ,
  `update_userid` int(11) NULL ,
  `is_delete` tinyint(4) NULL ,
  `call_mobile` varchar(30) NULL ,
  `call_mobile_enc` varchar(128) NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(64) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `create_time`, `market_order_id`)
PARTITION BY RANGE(`create_time`)
(PARTITION p202111 VALUES [('2021-10-01 00:00:00'), ('2021-11-01 00:00:00')),
PARTITION p202112 VALUES [('2021-11-01 00:00:00'), ('2021-12-01 00:00:00')),
PARTITION p202201 VALUES [('2022-01-01 00:00:00'), ('2022-02-01 00:00:00')),
PARTITION p202202 VALUES [('2022-02-01 00:00:00'), ('2022-03-01 00:00:00')))
DISTRIBUTED BY HASH(`market_order_id`) BUCKETS 10
PROPERTIES (
"replication_num" = "3",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2147483648",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_num" = "3",
"dynamic_partition.buckets" = "10",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.start_day_of_month" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);
CREATE TABLE `model_smt_zib_activity_customer_order` (
  `id` int(11) NOT NULL ,
  `create_time` datetime NULL ,
  `callid` varchar(32) NULL DEFAULT "" ,
  `batch_no` varchar(128) NULL DEFAULT "" ,
  `uniqueid` varchar(64) NULL DEFAULT "" ,
  `customer_mobile` varchar(16) NULL DEFAULT "" ,
  `activity_id` varchar(32) NULL DEFAULT "" ,
  `activity_name` varchar(128) NULL DEFAULT "" ,
  `activity_type` varchar(16) NULL DEFAULT "" ,
  `activity_source_system` varchar(16) NULL DEFAULT "" ,
  `activity_start_time` datetime NULL ,
  `activity_end_time` datetime NULL ,
  `product_code` varchar(32) NULL DEFAULT "" ,
  `customer_groupid` varchar(32) NULL DEFAULT "" ,
  `invite_id` varchar(32) NULL DEFAULT "" ,
  `invite_order` varchar(64) NULL DEFAULT "" ,
  `task_type` varchar(16) NULL DEFAULT "" ,
  `contract_no` varchar(32) NULL DEFAULT "" ,
  `offer_effective_begin_time` datetime NULL ,
  `offer_effective_end_time` datetime NULL ,
  `intellect_temp_id` varchar(32) NULL DEFAULT "" ,
  `intellect_call_start_time` datetime NULL ,
  `intellect_call_expand_param` varchar(4096) NULL ,
  `business_line` varchar(63) NULL DEFAULT "" ,
  `product_no` varchar(63) NULL DEFAULT "" ,
  `business_category` varchar(63) NULL DEFAULT "" ,
  `label_category` varchar(63) NULL DEFAULT "" ,
  `customer_group` varchar(63) NULL DEFAULT "" ,
  `status` tinyint(4) NULL DEFAULT "0" ,
  `order_status` tinyint(4) NULL DEFAULT "0" ,
  `is_delete` tinyint(4) NULL DEFAULT "0" ,
  `customer_mobile_enc` varchar(128) NULL DEFAULT "" ,
  `create_user_id` int(11) NULL ,
  `update_user_id` int(11) NULL ,
  `update_time` datetime NULL ,
  `etl_time` datetime NULL ,
  `etl_record_source` varchar(64) NULL ,
  `etl_record_type` int(11) NULL 
) ENGINE=OLAP
UNIQUE KEY(`id`, `create_time`, `callid`)
PARTITION BY RANGE(`create_time`)
(PARTITION p202108 VALUES [('0000-01-01 00:00:00'), ('2021-09-01 00:00:00')),
PARTITION p202109 VALUES [('2021-09-01 00:00:00'), ('2021-10-01 00:00:00')),
PARTITION p202110 VALUES [('2021-10-01 00:00:00'), ('2021-11-01 00:00:00')),
PARTITION p202111 VALUES [('2021-11-01 00:00:00'), ('2021-12-01 00:00:00')),
PARTITION p202112 VALUES [('2021-12-01 00:00:00'), ('2022-01-01 00:00:00')),
PARTITION p202201 VALUES [('2022-01-01 00:00:00'), ('2022-02-01 00:00:00')))
DISTRIBUTED BY HASH(`create_time`, `callid`) BUCKETS 8
PROPERTIES (
"replication_num" = "3",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "MONTH",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2147483648",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_num" = "3",
"dynamic_partition.buckets" = "8",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.start_day_of_month" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);

#############################################建视图语句######################################################
CREATE VIEW `dwd_telemarketing_cust_market_register_dtl_v` AS SELECT  `a`.`unique_id`                                                AS `unique_id` ,`a`.`customer_mobile_enc`                                      AS `customer_mobile_enc` ,`a`.`activity_id`                                              AS `activity_id` ,`a`.`activity_name`                                            AS `activity_name` ,`a`.`batch_no`                                                 AS `batch_no` ,`a`.`create_time`                                              AS `create_time` ,`b`.`apply_no`                                                 AS `apply_no` ,`b`.`apply_time`                                               AS `apply_time` ,`b`.`status`                                                   AS `status` ,`d`.`org_leader_name`                                          AS `org_name` ,`d`.`group_leader_name`                                        AS `group_name` ,`d`.`user_empno`                                               AS `user_empno` ,`d`.`user_name`                                                AS `user_name` ,get_json_string(`a`.`intellect_call_expand_param`,'$.user_id') AS `user_id` ,`d`.`effe_iden_time`                                           AS `effe_iden_time` ,`c`.`list_allocation_time`                                     AS `list_allocation_time` FROM ( SELECT  `a12`.`unique_id`                   AS `unique_id` ,`a11`.`customer_mobile_enc`         AS `customer_mobile_enc` ,`a11`.`batch_no`                    AS `batch_no` ,`a11`.`activity_id`                 AS `activity_id` ,`a11`.`activity_name`               AS `activity_name` ,`a11`.`create_time`                 AS `create_time` ,`a11`.`callid`                      AS `callid` ,`a11`.`intellect_call_expand_param` AS `intellect_call_expand_param` FROM `model_smt_zib_activity_customer_order` a11 LEFT OUTER JOIN ( SELECT  `a121`.`unique_id`  AS `unique_id` ,`a121`.`mobile_enc` AS `mobile_enc` FROM `model_resu_info` a121 WHERE `a121`.`unique_id` IS NOT NULL AND trim(`a121`.`unique_id`) != '' UNION ALL SELECT  `a122`.`unique_id`  AS `unique_id` ,`a122`.`mobile_enc` AS `mobile_enc` FROM `model_resu_info_back` a122 WHERE `a122`.`unique_id` IS NOT NULL AND trim(`a122`.`unique_id`) != '' ) a12 ON `a11`.`customer_mobile_enc` = `a12`.`mobile_enc` WHERE `a11`.`create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `a11`.`activity_source_system` = 'MANUAL' AND `a11`.`activity_type` IN ('ZCWSQ_CLM_SS', 'ZCWSQ_CLM_LX') AND (`a11`.`uniqueid` IS NULL OR trim(`a11`.`uniqueid`) = '') UNION ALL SELECT  `a2`.`uniqueid`                    AS `unique_id` ,`a2`.`customer_mobile_enc`         AS `customer_mobile_enc` ,`a2`.`batch_no`                    AS `batch_no` ,`a2`.`activity_id`                 AS `activity_id` ,`a2`.`activity_name`               AS `activity_name` ,`a2`.`create_time`                 AS `create_time` ,`a2`.`callid`                      AS `callid` ,`a2`.`intellect_call_expand_param` AS `intellect_call_expand_param` FROM `model_smt_zib_activity_customer_order` a2 WHERE `a2`.`create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `a2`.`activity_source_system` = 'MANUAL' AND `a2`.`activity_type` IN ('ZCWSQ_CLM_SS', 'ZCWSQ_CLM_LX') AND `a2`.`uniqueid` IS NOT NULL AND trim(`a2`.`uniqueid`) != '' UNION ALL SELECT  `a3`.`customer_id`         AS `unique_id` ,`a3`.`customer_mobile_enc` AS `customer_mobile_enc` ,`a3`.`batch_no`            AS `batch_no` ,`a3`.`activity_id`         AS `activity_id` ,`a3`.`activity_name`       AS `activity_name` ,`a3`.`create_time`         AS `create_time` ,`a3`.`callid`              AS `callid` ,''                         AS `intellect_call_expand_param` FROM `model_smt_zib_activity_customer` a3 WHERE `a3`.`create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `a3`.`order_type` = 'AI_TO_MANUAL' ) a LEFT OUTER JOIN ( SELECT  `unique_id`  AS `unique_id` ,`apply_no`   AS `apply_no` ,`apply_time` AS `apply_time` ,`status`     AS `status` FROM `model_yllpa_main` WHERE `apply_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) ) b ON `a`.`unique_id` = `b`.`unique_id` LEFT OUTER JOIN ( SELECT  `c1`.`id`          AS `id` ,`c1`.`callid`      AS `callid` ,`c1`.`activity_id` AS `activity_id` ,`c2`.`create_time` AS `list_allocation_time` FROM ( SELECT  `callid`          AS `callid` ,`id`              AS `id` ,`activity_id`     AS `activity_id` ,`market_order_id` AS `market_order_id` FROM `model_smt_zib_activity_customer` WHERE `create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `activity_name` REGEXP '未申请|注册未|非注册|MOT|注册' ) c1 INNER JOIN ( SELECT  `market_order_id` AS `market_order_id` ,`create_time`     AS `create_time` FROM ( SELECT  `market_order_id`                                                                         AS `market_order_id` ,`create_time`                                                                             AS `create_time` ,row_number() OVER (PARTITION BY `call_mobile_enc` ORDER BY `create_time` ASC NULLS FIRST) AS `row_order` FROM `model_smt_zib_assign_detail` WHERE `create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `user_id` != 99999999 ) c21 WHERE `c21`.`row_order` = 1 ) c2 ON `c1`.`market_order_id` = `c2`.`market_order_id` ) c ON `a`.`callid` = `c`.`callid` AND `a`.`activity_id` = `c`.`activity_id` LEFT OUTER JOIN ( SELECT  `d1`.`activity_customerid`                                                                                                       AS `activity_customerid` ,`user_empno`                                                                                                                     AS `user_empno` ,`user_name`                                                                                                                      AS `user_name` ,`d1`.`org_name_od`                                                                                                               AS `group_leader_name` ,CASE WHEN (`d3`.`org_leader_name` IS NULL OR `d3`.`org_leader_name` = '') AND `d1`.`org_name_od` = '' THEN '' WHEN `d3`.`org_leader_name` IS NULL OR `d3`.`org_leader_name` = '' THEN `d1`.`org_name_od`  ELSE `d3`.`org_leader_name` END AS `org_leader_name` ,CASE WHEN `action_first_code` IN ('C01','C02') THEN `d1`.`create_time`  ELSE '' END                                              AS `effe_iden_time` FROM ( SELECT  `activity_customerid` AS `activity_customerid` ,`create_time`         AS `create_time` ,`action_first_code`   AS `action_first_code` ,`org_name_od`         AS `org_name_od` ,`user_empno`          AS `user_empno` ,`user_name`           AS `user_name` ,`rnk`                 AS `rnk` FROM ( SELECT  `activity_customerid`                                                                             AS `activity_customerid` ,`create_time`                                                                                     AS `create_time` ,`action_first_code`                                                                               AS `action_first_code` ,`user_empno`                                                                                      AS `user_empno` ,`user_name`                                                                                       AS `user_name` ,CASE WHEN `org_name` LIKE '%（代）%' THEN substr(`org_name`,1,instr(`org_name`,'（') - 1)  ELSE substr(`org_name`,1,char_length(`org_name`) - 1) END AS `org_name_od` ,row_number() OVER (PARTITION BY `activity_customerid` ORDER BY `first_dial_time` ASC NULLS FIRST) AS `rnk` FROM `model_smt_zib_call_result` WHERE `create_time` >= date_sub(date_format(now(), '%Y-%m-%d'), 10) AND `user_empno` != 'ICC_yuce' AND `activity_name` REGEXP '未申请|注册未|非注册|MOT|注册' ) t WHERE `rnk` = 1 ) d1 LEFT OUTER JOIN ( SELECT  `icc_account` AS `icc_account` ,`name`        AS `group_leader_name` ,`org_name`    AS `group_name` FROM `model_smt_mdm_user` WHERE `position_id` = 4 AND `status` = 1 GROUP BY  `icc_account` ,`name` ,`org_name` ) d2 ON `d1`.`org_name_od` = `d2`.`group_leader_name` LEFT OUTER JOIN ( SELECT  `name`     AS `org_leader_name` ,`org_name` AS `org_name` FROM `model_smt_mdm_user` WHERE `position_id` = 5 AND `status` = 1 GROUP BY  `name` ,`org_name` ) d3 ON `d2`.`group_name` = `d3`.`org_name` ) d ON `c`.`id` = `d`.`activity_customerid`;

[morningman]

This bug has been fixed in #7931

[ChPi]

系统版本：0.14.0 Release
报错信息：
执行查询sql
具体表现：be挂了
操作系统：centos 7
是否能复现以及复现步骤：生产环境be挂了，同一个sql在其他环境执行成功，难以在其他环境复现，应该跟数据有关。后面doris升级到最新看看
相关Issue：
be info
fail to allocate mem via posix_memalign, res=12, errmsg=Cannot allocate memory
be out
@ 0x7f8efe205400 (unknown)
@ 0x7f8efe31bc40 __memcpy_ssse3
@ 0x11b25d1 doris::Tuple::deep_copy()
@ 0x17f44b3 doris::DataStreamSender::Channel::add_row()
@ 0x17f4acb doris::DataStreamSender::send()
@ 0x119b5c8 doris::PlanFragmentExecutor::open_internal()
@ 0x119bbc0 doris::PlanFragmentExecutor::open()
@ 0x111c18e doris::FragmentExecState::execute()
@ 0x111f4b6 doris::FragmentMgr::_exec_actual()
@ 0x112540c std::_Function_handler<>::_M_invoke()
@ 0x12aeef2 doris::ThreadPool::dispatch_thread()
@ 0x12a6d58 doris::Thread::supervise_thread()
@ 0x7f8efdfbaea5 start_thread
@ 0x7f8efe2cd96d __clone
core dump
#0 0x00007f357a4c1c40 in __memcpy_ssse3 () from /lib64/libc.so.6
#1 0x00000000011b25d1 in deep_copy (convert_ptrs=, pool=0x1fa901f40, desc=..., dst=,
this=) at /root/doris/be/src/runtime/tuple.cpp:76
#2 doris::Tuple::deep_copy (this=, desc=..., pool=0x1fa901f40, convert_ptrs=convert_ptrs@entry=false)
at /root/doris/be/src/runtime/tuple.cpp:59
#3 0x00000000017f44b3 in deep_copy (pool=, desc=..., this=)
at /root/doris/be/src/runtime/tuple.h:76
#4 doris::DataStreamSender::Channel::add_row (this=0x21ab738c0, row=row@entry=0x21a4dc000)
at /root/doris/be/src/runtime/data_stream_sender.cpp:262
#5 0x00000000017f4acb in doris::DataStreamSender::send (this=0x201c7f380, state=, batch=0x201f2e580)
at /root/doris/be/src/runtime/data_stream_sender.cpp:501
#6 0x000000000119b5c8 in doris::PlanFragmentExecutor::open_internal (this=this@entry=0x1fa64a370)
at /root/doris/be/src/runtime/plan_fragment_executor.cpp:310
#7 0x000000000119bbc0 in doris::PlanFragmentExecutor::open (this=this@entry=0x1fa64a370)
at /root/doris/be/src/runtime/plan_fragment_executor.cpp:253
#8 0x000000000111c18e in doris::FragmentExecState::execute (this=0x1fa64a300)
at /root/doris/be/src/runtime/fragment_mgr.cpp:219
#9 0x000000000111f4b6 in doris::FragmentMgr::_exec_actual(std::shared_ptrdoris::FragmentExecState, std::function<void (doris::PlanFragmentExecutor*)>) (this=0x65ed2c0, exec_state=..., cb=...)
at /root/doris/be/src/runtime/fragment_mgr.cpp:423
#10 0x000000000112540c in __invoke_impl<void, void (doris::FragmentMgr::&)(std::shared_ptrdoris::FragmentExecState, std::function<void(doris::PlanFragmentExecutor)>), doris::FragmentMgr*&, std::shared_ptrdoris::FragmentExecState&, std::function<void(doris::PlanFragmentExecutor*)>&> (__t=@0x219d463b0: 0x65ed2c0, __f=
@0x219d46370: (void (doris::FragmentMgr::)(doris::FragmentMgr * const, std::shared_ptrdoris::FragmentExecState, std::function<void(doris::PlanFragmentExecutor)>)) 0x111f490 <doris::FragmentMgr::_exec_actual(std::shared_ptrdoris::FragmentExecState, std::function<void (doris::PlanFragmentExecutor*)>)>) at /usr/include/c++/7.3.0/bits/invoke.h:73
---Type to continue, or q to quit---
#11 __invoke<void (doris::FragmentMgr::&)(std::shared_ptrdoris::FragmentExecState, std::function<void(doris::PlanFragmentExecutor)>), doris::FragmentMgr*&, std::shared_ptrdoris::FragmentExecState&, std::function<void(doris::PlanFragmentExecutor*)>&> (__fn=
@0x219d46370: (void (doris::FragmentMgr::)(doris::FragmentMgr * const, std::shared_ptrdoris::FragmentExecState, std::function<void(doris::PlanFragmentExecutor)>)) 0x111f490 <doris::FragmentMgr::_exec_actual(std::shared_ptrdoris::FragmentExecState, std::function<void (doris::PlanFragmentExecutor*)>)>) at /usr/include/c++/7.3.0/bits/invoke.h:95
#12 __call<void, 0, 1, 2> (__args=..., this=0x219d46370) at /usr/include/c++/7.3.0/functional:632
#13 operator()<> (this=0x219d46370) at /usr/include/c++/7.3.0/functional:718
#14 std::_Function_handler<void (), std::_Bind_result<void, void (doris::FragmentMgr::(doris::FragmentMgr, std::shared_ptrdoris::FragmentExecState, std::function<void (doris::PlanFragmentExecutor*)>))(std::shared_ptrdoris::FragmentExecState, std::function<void (doris::PlanFragmentExecutor*)>)> >::_M_invoke(std::_Any_data const&) (__functor=...)
at /usr/include/c++/7.3.0/bits/std_function.h:316
#15 0x00000000012aeef2 in operator() (this=0x1fa91cad8) at /usr/include/c++/7.3.0/bits/std_function.h:706
#16 run (this=0x1fa91cad0) at /root/doris/be/src/util/threadpool.cpp:42
#17 doris::ThreadPool::dispatch_thread (this=0x6706000)
at /root/doris/be/src/util/threadpool.cpp:548
#18 0x00000000012a6d58 in operator() (this=0x67d2018) at /usr/include/c++/7.3.0/bits/std_function.h:706
#19 doris::Thread::supervise_thread (arg=0x67d2000) at /root/doris/be/src/util/thread.cpp:385
#20 0x00007f357a160ea5 in start_thread () from /lib64/libpthread.so.0
#21 0x00007f357a47396d in clone () from /lib64/libc.so.6

[mylifetostart]

flink程序使用stream load导入doris，be节点有一个挂了，启动之后，发现flink程序不写入doris，也不报错。必须重启flink任务，数据又重新写入。是不是http连接，在重启be节点之后，没法再连接be?

[deng-jeffer]

flink程序使用stream load导入doris，be节点有一个挂了，启动之后，发现flink程序不写入doris，也不报错。必须重启flink任务，数据又重新写入。是不是http连接，在重启be节点之后，没法再连接be?

你好,最新我也遇到了这样的问题,请问你是这样的解决的呢?

[zyw8136]

系统版本：0.15.0

报错信息：
Doris的routine load稳定复现crash（librdkakfa导致的）

具体表现：
librdkakfa断掉连接，导致be挂。

操作系统：
centOS 7

平台：x86

我这里目前routine load一段时间就会出现这样的coredump：

[morningman]

挂掉的时候，kafka测是否有什么特殊的操作？这是个已知问题，但是并没有找到可以文档复现的操作步骤。。

[zyw8136]

Kafka当时就是在接收flink处理出来的数据和doris通过routine load消费数据，其他的特殊操作还真的没有做！但是每次的core dump都是提示“Closing connection due to nodename change”，但是这个操作当时没有做这个操作！

[lymaimei]

系统版本：1.1.0-rc05
报错信息：日志未发现明显异常
执行了什么操作导致的报错：
执行sql如下：
insert into t
select max(c1)..max(c100) from d inner join b on ..
group by d.gc1,d.gc2..
单线程单个sql执行时间约10s，同时启动16个线程提交sql（插入数据到不同的表）到fe后，fe无响应，之后客户端工具(HeidiSql)执行其他sql亦无响应
同时启动2个无异常，同时启动8个有几率出现(并发时单个sql执行时间约60s)，循环启动每个线程前都Sleep1秒无异常。

具体表现：fe无响应
fe_host:8030/Session 页面显示查询时间持续递增 超过20分钟未结束

操作系统：Ubuntu 18.04.3 LTS

机器配置：1FE 2BE 8核32G

平台：x86

可复现

[dataalive]

FE no response , JVM heap size 4G or ?

[lymaimei]

FE no response , JVM heap size 4G or ?

-Xmx20480m

[dataalive]

这个FE 无响应的时候可以打个jstack，或者加下微信：warpdrive1987 线下分析下。

[lymaimei]

Fixed,should be updated on 1.1.2

[wenwj0]

系统版本：1.0.0
数据：1.5T TPC-H(用tools/tpch-tools生成)

机器配置：CPU核数：44core、内存384G、4节点
平台：x86

操作：执行doris/tools/tpch-tools/queries/q1.sql等多条SQL；
参数配置：set exec_mem_limit=357698037760;
具体表现：
be进程挂掉，界面显示
Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason

dmesg -T 显示
[Tue Aug 9 11:46:05 2022] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/,task=doris_be,pid=1111674,uid=0

[Tue Aug 9 11:46:05 2022] Out of memory: Killed process 1111674 (doris_be) total-vm:25801980kB, anon-rss:17316216kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:39496kB oom_score_adj:0

[Tue Aug 9 11:46:06 2022] oom_reaper: reaped process 1111674 (doris_be), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

是否能复现：是
备注：相同的环境测试100G数据量的没有OOM报错；1.5T数据量有部分TPC-H的SQL能跑通。

[LOVEGISER]

这个问题应该在1.1修复了，本质是内存没限制住。升级到1.1现象应该是当大sql执行内存不足时直接提示内存不够。而不会引起be挂掉。

[wenwj0]

更换为1.1版本之后还是出现了OOM
[Fri Aug 12 09:38:20 2022] Out of memory: Kill process 39374 (doris_be) score 915 or sacrifice child
[Fri Aug 12 09:38:20 2022] Killed process 39374 (doris_be) total-vm:563596196kB, anon-rss:497038368kB, file-rss:0kB, shmem-rss:0kB

[dataalive]

这个我们比较奇怪的啊，内存384G，怎么这里提示都快500g了，方便远程的话加下微信：warpdrive1987 分析下

[lymaimei]

系统版本：1.2.0
报错信息：日志未发现明显异常
执行了什么操作导致的报错：
执行sql如下：
CREATE TABLE TP
(
PERIOD DATE,
startPeriod VARCHAR(10)
)
UNIQUE KEY(PERIOD, startPeriod)
DISTRIBUTED BY HASH(PERIOD) BUCKETS 10 PROPERTIES("replication_num" = "1")
CREATE TABLE TP2
(
PERIOD DATE,
startPeriod VARCHAR(10)
)
UNIQUE KEY(PERIOD, startPeriod)
DISTRIBUTED BY HASH(PERIOD) BUCKETS 10 PROPERTIES("replication_num" = "1")
TP 和 TP2 插入相同数据
SELECT * FROM TP2
INNER JOIN TP ON TP.PERIOD = TP2.PERIOD
无结果
具体表现：INNER JOIN ON DATE 无结果

操作系统：Ubuntu 18.04.3 LTS

机器配置：1FE 2BE 8核32G

平台：x86