Skip to content

Fix tag empty error and disorder timestamp. #489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 19, 2025
Merged

Conversation

ColinLeeo
Copy link
Contributor

@ColinLeeo ColinLeeo commented May 12, 2025

In this pr:
Fixed the issue of out-of-order data writes.
Fixed the issue related to writing and reading with empty tags.
Added support for writing null tag values.

@ColinLeeo ColinLeeo changed the title Fix tag empty error. Fix tag empty error and disorder timestamp. May 12, 2025
@codecov-commenter
Copy link

codecov-commenter commented May 12, 2025

Codecov Report

Attention: Patch coverage is 85.95041% with 17 lines in your changes missing coverage. Please review.

Project coverage is 65.77%. Comparing base (2995362) to head (486017c).
Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
cpp/src/common/tsblock/tsblock.cc 0.00% 4 Missing ⚠️
cpp/src/common/allocator/byte_stream.h 82.35% 3 Missing ⚠️
cpp/src/common/device_id.h 94.82% 3 Missing ⚠️
cpp/src/common/tsfile_common.h 60.00% 1 Missing and 1 partial ⚠️
cpp/src/reader/qds_with_timegenerator.cc 0.00% 2 Missing ⚠️
cpp/src/common/allocator/my_string.h 66.66% 1 Missing ⚠️
cpp/src/common/tablet.cc 90.90% 0 Missing and 1 partial ⚠️
...p/src/reader/block/single_device_tsblock_reader.cc 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #489      +/-   ##
===========================================
+ Coverage    65.70%   65.77%   +0.06%     
===========================================
  Files          566      566              
  Lines        33360    33430      +70     
  Branches      4664     4688      +24     
===========================================
+ Hits         21919    21987      +68     
- Misses       10789    10790       +1     
- Partials       652      653       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines 201 to 216
for (int32_t pos : id_column_context.pos_in_result_) {
common::String device_id(
device_query_task_->get_device_id()->get_segments().at(
id_column_context.pos_in_device_id_));
common::String device_id;
if (device_query_task_->get_device_id()->segment_num() <=
id_column_context.pos_in_device_id_) {
device_id = common::String("");
} else {
device_id = common::String(
device_query_task_->get_device_id()->get_segments().at(
id_column_context.pos_in_device_id_));
}
if (RET_FAIL(col_appenders_[pos + 1]->fill(
(char*)&device_id, sizeof(device_id),
current_block_->get_row_count()))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should distinguish null from empty string.
("table1", null, "tag1") is different from ("table1", "", "tag1")

@ColinLeeo ColinLeeo force-pushed the colin_fix_tag_empty branch from 3b74e14 to c40d043 Compare June 17, 2025 17:07
@ColinLeeo ColinLeeo requested a review from jt2594838 June 17, 2025 17:35
write_var_int(storage::NO_STR_TO_READ, out);
return ret;
}
size_t str_len = std::strlen(str);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not guaranteed that a tag will not contain '\0', so it is invalid to use std::strlen.

virtual const std::vector<std::string>& get_segments() const {
virtual const std::vector<char*>& get_segments() const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated, C-style strings may have a potential problem.
How about using std::string*?

Comment on lines 1083 to 1084
char *tmp_buf = static_cast<char *>(malloc(len + 1));
tmp_buf[len] = '\0';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is adding '\0' still necessary?

@jt2594838 jt2594838 merged commit af2a7ee into develop Jun 19, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants