Skip to content

path and to.dfs with R launched from two different directories #185

@nturenne

Description

@nturenne

Hi !

i use rmr2 and hadoop as a newbie on windows 7 with hadoop 2.7.1 and R 3.3.2

This are my parameters :

#init env R
Sys.setenv(HADOOP_CMD="D:/Soft/hadoop/bin/hadoop")
Sys.setenv(HADOOP_HOME="D:/Soft/hadoop/")
Sys.setenv("HADOOP_PREFIX"="D:/Soft/hadoop/")
Sys.setenv(HADOOP_STREAMING="D:/Soft/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar")

Sys.setenv(TMP = 'D:\Soft\hadoop')
library(rhdfs)
library(rmr2)
library(ravro)
.jinit()
hdfs.init()

At this point all is ok

if i run with the path on which R is installed all is ok :

getwd()
[1] "C:/Program Files/R/R-3.3.2/bin/x64"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
17/11/16 17:16:08 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib l
ibrary
17/11/16 17:16:08 INFO compress.CodecPool: Got brand-new compressor [.deflate]

BUT if i run on another path there a path problem somewhere and i dont know why

getwd()
[1] "D:/Utilisateurs/turenn"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
Le chemin d'accès spécifié est introuvable.
Warning message:
l'exécution de la commande 'D:/Soft/hadoop/bin/hadoop jar D:/Soft/hadoop/share/h
adoop/tools/lib/hadoop-streaming-2.7.1.jar loadtb /tmp/file30a0419d6021 < /Temp/
RtmpAL2rCq/file30a0240f658e' renvoie un statut 1

why such problem about path when calling the same Rterm from two different directories
did i miss something ?

thank you for help
Nico

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions