-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metrics based custom scheduler plugin #24439
base: master
Are you sure you want to change the base?
Metrics based custom scheduler plugin #24439
Conversation
4e47db2
to
83cab5c
Compare
<?xml version="1.0" encoding="UTF-8"?> | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to add the parent tag to make this module consistent with the others:
<parent>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-root</artifactId>
<version>0.291-SNAPSHOT</version>
</parent>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! (docs)
83cab5c
to
745c65b
Compare
745c65b
to
f1fc17d
Compare
@auden-woolfson can you PTAL? |
@@ -0,0 +1 @@ | |||
router-scheduler.name=metricsBased |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be better to set this as a default value in the code, and leave this file empty?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The property name (router-scheduler.name) is added in the property file so that clients implementing a new scheduler plugin can just update the value to the name of the new scheduler. Also since the Metric based scheduler is just a sample implementation, it may not be good to set it as the default value in code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok... maybe it would make more sense to add a new optional property to the existing router-config.json file ("custom-scheduler-name" or something similar) instead of creating an entirely new file for this one property. This way would also allow for customers to use multiple different custom schedulers in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There can be more properties in future for new schedulers, like cpu/memory thresholds related to specific schedulers, which can use this property file
Thanks for the release note! New release note guidelines. Please remove the manual PR link in the following format from the release note entries for this PR.
I have updated the Release Notes Guidelines to remove the examples of manually adding the PR link. |
Description
Metrics based custom scheduler plugin
Motivation and Context
Metrics based custom scheduler plugin. Schedules cluster with lowest number of queries (running + queued queries).
Support for implementation of third party custom schedulers to be used by Presto router.
Impact
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.