Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Dynamic Partitioning Generates Incorrect Partition Values with Extra Day Component #47962

Open
3 tasks done
Baymine opened this issue Feb 17, 2025 · 1 comment
Open
3 tasks done

Comments

@Baymine
Copy link
Contributor

Baymine commented Feb 17, 2025

Search before asking

  • I had searched in the issues and found no similar issues.

Version

doris-2.1.6

What's Wrong?

When creating a table with dynamic partitioning in Doris, the generated partitions include an extra day component (e.g., '20250201' instead of '202502'). This causes insertions with the expected year_month format to fail due to missing partitions.

What You Expected?

The partitions should be created with the year_month values as integers representing the year and month (e.g., '202502'), without the extra day component.

How to Reproduce?

  1. Create a table with dynamic partitioning set to partition by month.
CREATE TABLE `test_tbl` (
  `year_month` int(11) NOT NULL,
  `unique_product_id` varchar NOT NULL 
) ENGINE=OLAP
UNIQUE KEY(`year_month`,`unique_product_id`)
PARTITION BY RANGE(`year_month`)
(
 PARTITION p_initial VALUES LESS THAN (202201)
)
DISTRIBUTED BY HASH(`unique_product_id`) BUCKETS 16
PROPERTIES (
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "month",
"dynamic_partition.start" = "-25",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "16",
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "V2"
);

2.Execute SHOW CREATE TABLE to view the partition definitions.

CREATE TABLE `test_tbl` (
  `year_month` int NOT NULL,
  `unique_product_id` varchar(65533) NOT NULL
) ENGINE=OLAP
UNIQUE KEY(`year_month`, `unique_product_id`)
PARTITION BY RANGE(`year_month`)
(PARTITION p202502 VALUES [("20250201"), ("20250301")),
PARTITION p202503 VALUES [("20250301"), ("20250401")),
PARTITION p202504 VALUES [("20250401"), ("20250501")))
DISTRIBUTED BY HASH(`unique_product_id`) BUCKETS 16
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "month",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-25",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_allocation" = "tag.location.default: 3",
"dynamic_partition.buckets" = "16",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.reserved_history_periods" = "NULL",
"dynamic_partition.storage_policy" = "",
"dynamic_partition.start_day_of_month" = "1",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"enable_unique_key_merge_on_write" = "true",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728",
"enable_mow_light_delete" = "false"
);
  1. Attempt to insert data with a year_month value expected to fit into a created partition.
insert into test_tbl (year_month, unique_product_id) values ('202503', '1232');
  1. Observe the error indicating no partition exists for the tuple.
Reason: no partition for this tuple.

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@w517424787
Copy link

应该是没控制好range字段类型,文档上说只允许用date/datetime类型创建,你这int类型都创建成功了,这里肯定是有bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants