分桶数据倾斜 #52337
分桶数据倾斜
#52337
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
doris2.1.9使用UNIQUE KEY
Idbigint NOT NULL作为分桶键DISTRIBUTED BY HASH(Id) BUCKETS 8,仍然产生了数据倾斜的情况"BucketIdx","AvgRowCount","AvgDataSize","Graph","Percent"
"0","39470","3485395",>>>>>>>>>>>>>>>>,"16.88 %"
"1","30235","2690570",>>>>>>>>>>>>>,"13.03 %"
"2","29866","2658435",>>>>>>>>>>>>,"12.87 %"
"3","25049","2237446",>>>>>>>>>>,"10.84 %"
"4","26317","2349757",>>>>>>>>>>>,"11.38 %"
"5","28558","2545532",>>>>>>>>>>>>,"12.33 %"
"6","28685","2542321",>>>>>>>>>>>>,"12.31 %"
"7","24129","2140665",>>>>>>>>>>,"10.37 %"
上官网查询到分桶策略
Hash 分桶:通过计算分桶列值的 crc32 哈希值,并对分桶数取模,将数据行均匀分布到分片中。
于是我使用 crc32(id)%8 发现其数据应该是分布均匀的,
"bucket_no","count(1)"
0,9549
1,9601
2,9486
3,9518
4,9452
5,9488
6,9567
7,9573
为什么会出现这种情况
Beta Was this translation helpful? Give feedback.
All reactions