You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jun 26, 2020. It is now read-only.
If you would like to write custom YARN application, i.e. via Spring Boot, using this manual (https://spring.io/guides/gs/yarn-basic/) you will face couple of errors:
One issue about guide.
Make sure you have corresponding versions of "spring-yarn-boot", "spring-boot-starter-parent" and "spring-boot-maven-plugin" in pom-files. Simpliest solution go to maven central and use latest versions.
Otherwise you may face "cannot instantiate 'yarnClient' bean" error, which tell you almost nothing why it failed
Another about this docker image.
Make sure you set limitless virtual memory for YARN.
For example, you may set in yarn-site.xml this properties:
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
<description>Whether virtual memory limits will be enforced for containers</description>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>4</value>
<description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
</property>
Cause you use it in Docker, you have to change it somehow after hadoop cluster initiation, or before, mapping volumes with "-v" option
Another solution is to use fork of this docker image, which has this settings by default, i.e. - https://github.com/sfedyakov/hadoop-271-cluster
The text was updated successfully, but these errors were encountered:
If you would like to write custom YARN application, i.e. via Spring Boot, using this manual (https://spring.io/guides/gs/yarn-basic/) you will face couple of errors:
Make sure you have corresponding versions of "spring-yarn-boot", "spring-boot-starter-parent" and "spring-boot-maven-plugin" in pom-files. Simpliest solution go to maven central and use latest versions.
Otherwise you may face "cannot instantiate 'yarnClient' bean" error, which tell you almost nothing why it failed
Make sure you set limitless virtual memory for YARN.
For example, you may set in yarn-site.xml this properties:
Cause you use it in Docker, you have to change it somehow after hadoop cluster initiation, or before, mapping volumes with "-v" option
Another solution is to use fork of this docker image, which has this settings by default, i.e. - https://github.com/sfedyakov/hadoop-271-cluster
The text was updated successfully, but these errors were encountered: