spatialhadoop2 icon indicating copy to clipboard operation
spatialhadoop2 copied to clipboard

Intermediate files when running index

Open stev-0 opened this issue 9 years ago • 2 comments

I am running the index command and it seems to be saving intermediate files into hdfs://user/filename. However spatialhadoop2 is looking for it in hdfs://user/my.username/filename. This is occuring at edu.umn.cs.spatialHadoop.operations.FileMBR.fileMBRMapReduce(FileMBR.java:252). This could well be something to do with the hadoop setup, but thought I would raise this here in case not.

stev-0 avatar Aug 29 '15 18:08 stev-0

I temporarily solved this by adding the full absolute path (e.g. outputPath = new Path("/user/myusername/"+inFiles[0].getName()+".mbr_"+(int)(Math.random()*1000000)); to line 334 of FileMBR,java, but this obviously isn't a generic solution.

stev-0 avatar Aug 30 '15 14:08 stev-0

SpatialHadoop uses the current working directory to place these intermediate files. This is typically the home directory of the user running the command. I don't understand why it tried to save to /user/filename instead of /user/username/filename. Did you change the default settings of the current working directory?

aseldawy avatar Sep 10 '15 04:09 aseldawy