Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-36946][Metrics] Optimize sink operator name truncate #25832

Merged
merged 3 commits into from
Jan 10, 2025

Conversation

Tartarus0zm
Copy link
Contributor

What is the purpose of the change

Avoid truncation of the Writer/Committer keyword.

Brief change log

  • add MetricUtil#truncateOperatorName to optimize Operator name

Verifying this change

  • SinkMetricsITCase#testMetrics

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Dec 20, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@1996fanrui 1996fanrui self-assigned this Dec 20, 2024
Copy link
Member

@1996fanrui 1996fanrui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Tartarus0zm for the optimization!

LGTM assuming CI is green.

@@ -366,6 +369,17 @@ private static void instantiateCPUMetrics(MetricGroup metrics) {
}
}

public static String truncateOperatorName(String operatorName, int maxLength) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we have a unit test for this method please.

Copy link
Contributor

@davidradl davidradl Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if maxlength is less than the Suffix length and there is a suffix in the data, this will end up with a negative length. Or is this not possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we have a unit test for this method please.

good catch

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if maxlength is less than the Suffix length and there is a suffix in the data, this will end up with a negative length. Or is this not possible?

Thanks @davidradl for the comment, it's interesting. IIUC, current PR only has 2 suffixes: Writer and Committer. Both of length are less than the maxlength(80), so it doesn't happen.

But I'm not sure will it happen in the future. Anyway, we could optimize the logic to avoid the negative length. It will make the code more robust.

@Tartarus0zm WDYT?

BTW, it's better to let reviewer click the resolve conversation. It's helpful for reviewer to check whether all comments are addressed properly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1996fanrui thanks for your remind. I has re-open the conversation.

Also, I've updated this PR, MetricUtils#truncateOperatorName internally does the length judgement of the OperatorName.
cc @davidradl

Copy link
Member

@1996fanrui 1996fanrui Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you pushed your code to github repo? The PR isn't updated, the latest change was 2 weeks ago.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1996fanrui yes, I updated this PR two weeks ago.
image
a. optimized MetricUtils#truncateOperatorName, internal judgement of how OperatorName should be truncated.
b. add Test MetricUtilsTest.java for MetricUtils#truncateOperatorName.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1996fanrui @davidradl hello , I update this PR again.
please take a look, thanks

Copy link
Contributor

@davidradl davidradl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

@Tartarus0zm
Copy link
Contributor Author

see comments

thanks for your review, I'll update the commit to addressing your concerns.

@Tartarus0zm
Copy link
Contributor Author

@1996fanrui @davidradl hi, I has updated the commit, take a look again?

@1996fanrui 1996fanrui merged commit b75d76e into apache:master Jan 10, 2025
@Tartarus0zm Tartarus0zm deleted the op_name_opts branch January 10, 2025 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants