Skip to content

Add monitorType JMX #472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
willmostly merged 1 commit into from
Feb 5, 2025
Merged

Add monitorType JMX #472

willmostly merged 1 commit into from
Feb 5, 2025

Conversation

@alaturqua
Copy link
Contributor

@alaturqua alaturqua commented Sep 17, 2024 •
edited by willmostly
Loading

Description

This pull request introduces a new monitorType for JMX-based monitoring in Trino Gateway. It includes documentation and configuration details for retrieving cluster information using specific JMX metrics.

Details

  • Component: JMX Monitoring in Trino Gateway

  • New Feature: Added support for monitorType: JMX to enable cluster monitoring via JMX metrics.

  • Metrics Used:

    • trino.execution:name=QueryManager
    • trino.metadata:name=DiscoveryNodeManager

  • Configuration Instructions:

    • Users can configure a username and password using backendState.
    • Credentials must be valid across all clusters with read rights on system_information.

  • Example Configuration:

    backendState:
  username: "user"
  password: "password"
clusterStatsConfiguration:
  monitorType: JMX

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

* Monitor backend health via JMX

@cla-bot cla-bot bot added the cla-signed label Sep 17, 2024
@alaturqua
Copy link
Contributor Author

alaturqua commented Sep 17, 2024

I am not very experienced with Java and Testing in Java. Please feel free to add tests.

@alaturqua alaturqua requested a review from mosabua September 17, 2024 23:49
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from 512ce06 to 82cf53d Compare September 17, 2024 23:59
@alaturqua alaturqua changed the title Add monitorType JMX via trino's /v1/jmx/mbean endpoint Add monitorType JMX Sep 17, 2024
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from 82cf53d to 40cb273 Compare September 18, 2024 00:02
@ebyhr ebyhr requested a review from willmostly September 24, 2024 05:00
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from 40cb273 to 5389a0d Compare September 25, 2024 07:27
@alaturqua alaturqua requested a review from ebyhr September 25, 2024 07:29
ebyhr
ebyhr reviewed Sep 27, 2024
View reviewed changes
Copy link
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just skimmed.

@alaturqua
Copy link
Contributor Author

alaturqua commented Oct 14, 2024

Thanks for the reviews. I was on vacation and didn't have time to implement the requested changes yet. I will take care of them in the next days.

I might need some support to activate the /v1/jmx/mbean endpoint in the test environment for testing.

@willmostly
Copy link
Contributor

willmostly commented Oct 29, 2024

@alaturqua please ping me when this is ready for another look

@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch 5 times, most recently from fb4ac57 to b333076 Compare November 17, 2024 21:46
@alaturqua alaturqua closed this Nov 17, 2024
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from b333076 to a11e5a4 Compare November 17, 2024 21:49
@alaturqua
Copy link
Contributor Author

alaturqua commented Nov 17, 2024

I had to sync my fork to be able to change my PR. But it got automatically closed, when I did so.
I am reopening.

@alaturqua alaturqua reopened this Nov 17, 2024
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from 6aebdfb to 995229a Compare November 17, 2024 22:23
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch 3 times, most recently from a79acde to 7671694 Compare November 20, 2024 09:07
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from 0abf92c to df3f071 Compare January 25, 2025 14:10
@alaturqua alaturqua requested a review from willmostly January 25, 2025 14:22
@alaturqua
Copy link
Contributor Author

alaturqua commented Jan 25, 2025

@willmostly

addressed all comments, ptal.

I didn't change the part, where @ebyhr had reacted with a thumb down.

ebyhr
ebyhr reviewed Feb 3, 2025
View reviewed changes
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch 3 times, most recently from 17bcc60 to ebd3b4f Compare February 3, 2025 05:45
alaturqua
alaturqua commented Feb 3, 2025
View reviewed changes
Copy link
Contributor Author

@alaturqua alaturqua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebyhr

Thank you for taking time and reviewing. I adjusted the points. Please take a look again.

gateway-ha/src/main/java/io/trino/gateway/ha/clustermonitor/ClusterStatsJmxMonitor.java Outdated
{
try {
Map<String, Integer> stats = response.attributes().stream()
.filter(attr -> "QueuedQueries".equals(attr.name()) ||
Copy link
Contributor Author

@alaturqua alaturqua Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@alaturqua alaturqua self-assigned this Feb 3, 2025
@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch 2 times, most recently from e4ae9be to 8a00bb7 Compare February 3, 2025 06:49
alaturqua
alaturqua commented Feb 3, 2025
View reviewed changes
Copy link
Contributor Author

@alaturqua alaturqua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebyhr

Thanks again, seems like I had overseen things. I changed them as well. Can you please take a look again?

@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch 2 times, most recently from 4f38500 to fc3bfd3 Compare February 3, 2025 07:23
willmostly
willmostly requested changes Feb 3, 2025
View reviewed changes
Copy link
Contributor

@willmostly willmostly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some cleanup

@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from f1eeb61 to 07e67f6 Compare February 3, 2025 22:54
alaturqua
alaturqua commented Feb 3, 2025
View reviewed changes
Copy link
Contributor Author

@alaturqua alaturqua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@willmostly

Thanks for your feedback and reviewing. I did the changes, you commented. Can you please take a look again?

@mosabua
Copy link
Member

mosabua commented Feb 5, 2025

@willmostly and @alaturqua discussed in the dev sync today and @willmostly will do a final review and merge

willmostly
willmostly requested changes Feb 5, 2025
View reviewed changes
@alaturqua
Copy link
Contributor Author

alaturqua commented Feb 5, 2025

@willmostly
I have addressed your comments. Please take a look again.

willmostly
willmostly approved these changes Feb 5, 2025
View reviewed changes
Copy link
Contributor

@willmostly willmostly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@alaturqua alaturqua force-pushed the ii-add-monitortype-jmx branch from e0e088f to 3e4a6fb Compare February 5, 2025 17:15
@willmostly willmostly merged commit 908cec3 into trinodb:main Feb 5, 2025
2 checks passed
@github-actions github-actions bot added this to the 14 milestone Feb 5, 2025
ebyhr
ebyhr reviewed Feb 5, 2025
View reviewed changes
gateway-ha/src/main/java/io/trino/gateway/ha/clustermonitor/ClusterStatsJmxMonitor.java
Comment on lines +146 to +147
JsonNode response = client.execute(preparedRequest, JMX_JSON_RESPONSE_HANDLER);
return Optional.ofNullable(response).map(JmxResponse::fromJson);
Copy link
Member

@ebyhr ebyhr Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is response really nullable?

@alaturqua
Copy link
Contributor Author

alaturqua commented Feb 5, 2025

@ebyhr

thanks for your feedback. I will open another PR to address your comments.

@alaturqua
Copy link
Contributor Author

alaturqua commented Feb 6, 2025

@ebyhr

Addressed your concerns in a new PR:

#611

Please take a look there.

@alaturqua alaturqua mentioned this pull request Feb 6, 2025
Merged
@mosabua mosabua mentioned this pull request Feb 11, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ebyhr ebyhr ebyhr left review comments

@willmostly willmostly willmostly approved these changes

@mosabua mosabua Awaiting requested review from mosabua

@andythsu andythsu Awaiting requested review from andythsu

Assignees

@alaturqua alaturqua

Labels

cla-signed

Milestone

14

Development

Successfully merging this pull request may close these issues.

5 participants

@alaturqua @willmostly @mosabua @ebyhr @andythsu