Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add resource_group to rw_streaming_jobs #20207

Merged
merged 4 commits into from
Jan 21, 2025

Conversation

shanicky
Copy link
Contributor

@shanicky shanicky commented Jan 19, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR creates a new rw_streaming_jobs system catalog based on the existing rw_table_fragments, adding the name field and resource_group field to display resource groups. By default, this field should be inherited from database. Additionally, this PR includes mock tests to verify the correctness of the ALTER RESET RESOURCE_GROUP command.

As many test scripts depend on rw_table_fragments, it will be deprecated in a future PR and replaced with rw_streaming_jobs.

dev=> create materialized view m as select * from t;
CREATE_MATERIALIZED_VIEW
dev=> alter materialized view m set resource_group to test;
ALTER_MATERIALIZED_VIEW
dev=> select * from rw_streaming_jobs;
 job | name | status  | parallelism | max_parallelism | resource_group
-----+------+---------+-------------+-----------------+----------------
   6 | t    | CREATED | ADAPTIVE    |             256 | default
   7 | m    | CREATED | ADAPTIVE    |             256 | test
(2 rows)

dev=> alter materialized view m reset resource_group;
ALTER_MATERIALIZED_VIEW
dev=> select * from rw_streaming_jobs;
 job | name | status  | parallelism | max_parallelism | resource_group
-----+------+---------+-------------+-----------------+----------------
   6 | t    | CREATED | ADAPTIVE    |             256 | default
   7 | m    | CREATED | ADAPTIVE    |             256 | default
(2 rows)

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.

Copy link

gru-agent bot commented Jan 20, 2025

This pull request has been modified. If you want me to regenerate unit test for any of the files related, please find the file in "Files Changed" tab and add a comment @gru-agent. (The github "Comment on this file" feature is in the upper right corner of each file in "Files Changed" tab.)

@shanicky shanicky force-pushed the peng/resource_group_system_table branch from e216931 to c74a026 Compare January 20, 2025 08:27
@yezizp2012
Copy link
Member

As many test scripts depend on rw_table_fragments, it will be deprecated in a future PR and replaced with rw_streaming_jobs.

Please also confirm if rw_table_fragments is not used on the cloud side, and if it is, we need to replace it as well.

Copy link
Member

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shanicky shanicky enabled auto-merge January 21, 2025 07:07
@shanicky shanicky disabled auto-merge January 21, 2025 07:10
@arkbriar
Copy link
Contributor

LGTM on the design. cc. @CAJan93 to take a look.

@shanicky shanicky force-pushed the peng/resource_group_system_table branch from 80ce249 to 35ea07f Compare January 21, 2025 08:29
@shanicky shanicky added this pull request to the merge queue Jan 21, 2025
Merged via the queue into main with commit 509ec7a Jan 21, 2025
34 checks passed
@shanicky shanicky deleted the peng/resource_group_system_table branch January 21, 2025 10:08
@shanicky shanicky changed the title feat: add specific_resource_group to rw_streaming_jobs feat: add resource_group to rw_streaming_jobs Jan 22, 2025
@lmatz lmatz added the user-facing-changes Contains changes that are visible to users label Mar 28, 2025
Copy link
Contributor

Hi, there.

📝 Telemetry Reminder:
If you're implementing this feature, please consider adding telemetry metrics to track its usage. This helps us understand how the feature is being used and improve it further.
You can find the function report_event of telemetry reporting in the following files. Feel free to ask questions if you need any guidance!

  • src/frontend/src/telemetry.rs
  • src/meta/src/telemetry.rs
  • src/stream/src/telemetry.rs
  • src/storage/compactor/src/telemetry.rs
    Or calling report_event_common (src/common/telemetry_event/src/lib.rs) as if finding it hard to implement.
    ✨ Thank you for your contribution to RisingWave! ✨

This is an automated comment created by the peaceiris/actions-label-commenter. Responding to the bot or mentioning it won't have any effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants