Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating the recommendation for Executor GC heuristic in case of high executor GC #442

Open
wants to merge 1 commit into
base: tuning_20190221
Choose a base branch
from
Open

Updating the recommendation for Executor GC heuristic in case of high executor GC #442

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions app/com/linkedin/drelephant/spark/heuristics/ExecutorGcHeuristic.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,13 @@ class ExecutorGcHeuristic(private val heuristicConfigurationData: HeuristicConfi

//adding recommendations to the result, severityTimeA corresponds to the ascending severity calculation
if (evaluator.severityTimeA.getValue > Severity.LOW.getValue) {
resultDetails = resultDetails :+ new HeuristicResultDetails("Gc ratio high", "The job is spending too much time on GC. We recommend increasing the executor memory.")
resultDetails = resultDetails :+ new HeuristicResultDetails("Gc ratio high",
"The job is spending too much time on GC. Recommended to increase the executor memory." + evaluator.parallelGcRecommendation + "Can also try reducing number of UDF calls.")
}
//severityTimeD corresponds to the descending severity calculation
if (evaluator.severityTimeD.getValue > Severity.LOW.getValue) {
resultDetails = resultDetails :+ new HeuristicResultDetails("Gc ratio low", "The job is spending too little time in GC. Please check if you have asked for more executor memory than required.")
resultDetails = resultDetails :+ new HeuristicResultDetails("Gc ratio low",
"The job is spending too little time in GC. Please check if you have asked for more executor memory than required.")
}

val result = new HeuristicResult(
Expand Down Expand Up @@ -103,6 +105,11 @@ object ExecutorGcHeuristic {
throw new Exception("No executor information available.")
}

val sparkExecutorExtraJavaOptions = appConfigurationProperties.getOrElse("spark.executor.extraJavaOptions","")
val isParallelGCEnabled: Boolean = sparkExecutorExtraJavaOptions.contains("XX:+UseParallelGC")
val isG1GCenabled: Boolean = sparkExecutorExtraJavaOptions.contains("XX:+UseG1GC")
val gcRecommendation: String = if (isParallelGCEnabled || isG1GCenabled) "" else "Enable ParallelGC or G1GC using spark.executor.extraJavaOptions."

lazy val appConfigurationProperties: Map[String, String] =
data.appConfigurationProperties
var (jvmTime, executorRunTimeTotal) = getTimeValues(executorSummaries)
Expand Down
10 changes: 9 additions & 1 deletion app/views/help/spark/helpExecutorGcHeuristic.scala.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,12 @@
<p>This analysis shows how much time a job is spending in GC. To normalise the results across all jobs, the ratio of the time a job spends in Gc to the total run time of the job is calculated. </p>
<p>A job is flagged if the ratio is too high, meaning the job spends too much time in GC.</p>
<h3>Suggestions</h3>
<p>We recommend increasing the executor memory.</p>
<ul>
<li>We recommend increasing the executor memory.</li>
<li>Enabling G1GC or ParallelGC using spark.executor.extraJavaOptions could help.</li>
<ul>
<li>User can enable G1GC or ParallelGC by adding <b>-XX:+UseG1GC</b> or <b>-XX:+UseParallelGC</b> respectively to Spark configuration spark.executor.extraJavaOptions</li>
</ul>
<li>High GC can occur if the number of UDF calls made is high, especially if the UDFs are inefficient or use a lot of memory.</li>
</ul>
<p>For some general guideline about how to tune GC for your Spark application refer <a href="https://spark.apache.org/docs/latest/tuning.html#garbage-collection-tuning" target="_blank">here</a></p>