-
Notifications
You must be signed in to change notification settings - Fork 99
Open
Description
Trying to import MOD09GQ.005 file in Spatial Hadoop 2.3 (installation shipped with apache hadoop).
File was taken from here:
http://e4ftl01.cr.usgs.gov/MOLT/MOD09GQ.005/2016.02.17/MOD09GQ.A2016048.h35v09.005.2016050064358.hdf
File was copied to HDFS as 1.hdf. Index command:
bin/shadoop index 1.hdf 1.grid mbr:0,0,100,100 sindex:rtree shape:nasapoint
returns the following error:
bin/shadoop index 1.hdf 1.grid mbr:0,0,100,100 sindex:rtree shape:nasapoint
16/03/18 19:13:09 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/03/18 19:13:09 WARN snappy.LoadSnappy: Snappy native library not loaded
16/03/18 19:13:09 INFO mapred.FileInputFormat: No block filter specified
16/03/18 19:13:09 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/18 19:13:09 INFO operations.Repartition: Reading a sample of 10%
16/03/18 19:13:09 INFO mapred.FileInputFormat: No block filter specified
16/03/18 19:13:09 INFO mapred.FileInputFormat: Total input paths to process : 1
16/03/18 19:13:10 INFO mapred.JobClient: Running job: job_local161844959_0001
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Waiting for map tasks
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Starting task: attempt_local161844959_0001_m_000000_0
16/03/18 19:13:10 INFO util.ProcessTree: setsid exited with exit code 0
16/03/18 19:13:10 INFO mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5a4ca8a8
16/03/18 19:13:10 INFO mapred.MapTask: Processing split: file:/root/spatialhadoop-2.3-hadoop-1.2.1/1.hdf:0+71498628
16/03/18 19:13:10 INFO mapred.SpatialRecordReader: Open a SpatialRecordReader to file: file:/root/spatialhadoop-2.3-hadoop-1.2.1/1.hdf
16/03/18 19:13:10 INFO mapred.MapTask: numReduceTasks: 1
16/03/18 19:13:10 INFO mapred.MapTask: io.sort.mb = 100
16/03/18 19:13:10 INFO mapred.MapTask: data buffer = 79691776/99614720
16/03/18 19:13:10 INFO mapred.MapTask: record buffer = 262144/327680
16/03/18 19:13:10 INFO mapred.LocalJobRunner: Map task executor complete.
16/03/18 19:13:10 WARN mapred.LocalJobRunner: job_local161844959_0001
java.lang.Exception: java.lang.NumberFormatException: empty String
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
Caused by: java.lang.NumberFormatException: empty String
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1011)
at java.lang.Double.parseDouble(Double.java:540)
at edu.umn.cs.spatialHadoop.io.TextSerializerHelper.consumeDouble(TextSerializerHelper.java:182)
at edu.umn.cs.spatialHadoop.core.Point.fromText(Point.java:136)
at edu.umn.cs.spatialHadoop.nasa.NASAPoint.fromText(NASAPoint.java:77)
at edu.umn.cs.spatialHadoop.operations.Sampler$Map.map(Sampler.java:122)
at edu.umn.cs.spatialHadoop.operations.Sampler$Map.map(Sampler.java:69)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
16/03/18 19:13:11 INFO mapred.JobClient: map 0% reduce 0%
16/03/18 19:13:11 INFO mapred.JobClient: Job complete: job_local161844959_0001
16/03/18 19:13:11 INFO mapred.JobClient: Counters: 0
16/03/18 19:13:11 INFO mapred.JobClient: Job Failed: NA
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
at edu.umn.cs.spatialHadoop.operations.Sampler.sampleMapReduceWithRatio(Sampler.java:211)
at edu.umn.cs.spatialHadoop.operations.Sampler.sample(Sampler.java:543)
at edu.umn.cs.spatialHadoop.operations.Repartition.packInRectangles(Repartition.java:494)
at edu.umn.cs.spatialHadoop.operations.Repartition.packInRectangles(Repartition.java:463)
at edu.umn.cs.spatialHadoop.operations.Repartition.repartitionLocal(Repartition.java:590)
at edu.umn.cs.spatialHadoop.operations.Repartition.repartition(Repartition.java:676)
at edu.umn.cs.spatialHadoop.operations.Repartition.main(Repartition.java:721)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at edu.umn.cs.spatialHadoop.operations.Main.main(Main.java:115)
Metadata
Metadata
Assignees
Labels
No labels