spatialhadoop2 icon indicating copy to clipboard operation
spatialhadoop2 copied to clipboard

who can tell me spets to install the spatialhadoop on the hadoop?

Open GISerCAU opened this issue 9 years ago • 20 comments

Just like the title!

GISerCAU avatar Apr 21 '15 04:04 GISerCAU

  1.          Download Spatial Hadoop tar file on existing Hadoop cluster machine
    
  2.          Untar the downloaded SpatialHadoop bundle
    
  3.          Copy SpatialHadoop/conf/\* folder contents to $HADOOP_HOME/conf/ directory
    
  4.          Copy SpatialHadoop/bin/\* folder contents to $HADOOP_HOME/bin/ directory
    
  5.          Copy SpatialHadoop/lib/\* folder contents to $HADOOP_HOME/lib/ directory
    
  6.          Copy spatialhadoop-x.x/spatialhadoop-main.jar file parallel to Hadoop lib folder, i.e., $HADOOP_HOME/\* 
    

CreativeCreamer avatar Apr 21 '15 06:04 CreativeCreamer

Thank you, C. C. i did this fellow the six steps; but have one error: Error: Could not find or load main class edu.umn.cs.spatialHadoop.operations.Main

Are there any documents need me to configure? Thanks!

bigclove avatar Apr 21 '15 07:04 bigclove

Have you tried running any command after configuring spatialHadoop ? If yes, then post the command as well.

CreativeCreamer avatar Apr 21 '15 08:04 CreativeCreamer

OK! JUST IT! thanks! guest@master:/usr/hadoop/bin$ /usr/hadoop/bin/shadoop Error: Could not find or load main class edu.umn.cs.spatialHadoop.operations.Main

bigclove avatar Apr 21 '15 08:04 bigclove

Have you restarted the hadoop services after spatialhadoop configuration ?

CreativeCreamer avatar Apr 21 '15 08:04 CreativeCreamer

yes! the hadoop version is 2.4.0. it is ok?

bigclove avatar Apr 21 '15 08:04 bigclove

I think its not able to find the "spatialhadoop-main.jar" file in HADOOP_HOME directory parallel to $HADOOP_HOME/bin folder. I have tried with Hadoop 2.2.0 and spatialhadoop-2.2 version, and above steps are working on that.

CreativeCreamer avatar Apr 21 '15 08:04 CreativeCreamer

can i give you an email about the details i did? thanks. post your email address!

bigclove avatar Apr 21 '15 09:04 bigclove

Post the steps here whatever you tried.

CreativeCreamer avatar Apr 21 '15 10:04 CreativeCreamer

Step 1 1、Download Spatial Hadoop tar file on existing Hadoop cluster machine 2、Untar the downloaded SpatialHadoop bundle As the fig.1, left is the files of SpatialHadoop, right is the $HADOOP_HOME/*

1

Step 2 1、Copy SpatialHadoop/conf/* folder contents to $HADOOP_HOME/conf/ directory 2、Copy SpatialHadoop/bin/* folder contents to $HADOOP_HOME/bin/ directory 3、Copy SpatialHadoop/lib/* folder contents to $HADOOP_HOME/lib/ directory 4、Copy spatialhadoop-x.x/spatialhadoop-main.jar file parallel to Hadoop lib folder, i.e., $HADOOP_HOME/*

2

After finished: /bin/shadoop /usr/hadoop/bin/

3

/conf/spatial-site.xml /usr/hadoop/etc/hadoop/

4

bigclove avatar Apr 22 '15 01:04 bigclove

Next!

lib 5 Spatialhadoop-main.jar 6 Then restart the hadoop! 7 Then command /usr/hadoop/bin/shadoop _en se pb1poinjhi_6yrc

bigclove avatar Apr 22 '15 01:04 bigclove

Check the entry for Main class in 'bin/shadoop' file and the place where your Main.class in present.

a

Main.class is in spatialhadoop-main.jar -> \edu\umn\cs\spatialHadoop\operations\Main.class

CreativeCreamer avatar Apr 22 '15 09:04 CreativeCreamer

now it is not ok.

image

image

bigclove avatar Apr 22 '15 10:04 bigclove

Not sure.. Why it is throwing exception for rangeQuery ? You can try running "hadoop jar spatialhadoop-main.jar mbr" It should asks for "other parameters like inputfile".

CreativeCreamer avatar Apr 22 '15 10:04 CreativeCreamer

ok , i think i will try other version. thanks

bigclove avatar Apr 23 '15 07:04 bigclove

It might be a little late but you need to get the right SpatialHadoop package based on your Hadoop version. For Hadoop 1.x, you need to download the file http://spatialhadoop.cs.umn.edu/downloads/spatialhadoop-2.3.tar.gz For Hadoop 2.x (as in your case), you need to download an alternative version as the directory structure of Hadoop 2.x is different. Download the file http://spatialhadoop.cs.umn.edu/downloads/spatialhadoop-2.3-2.tar.gz You can also generate these files from the source code by running the ant task ant package1 or ant package2

Then, you need to extract the package (.tar.gz) file into your Hadoop home directory ($HADOOP_HOME). It will automatically place each file in the correct path. You can do this by running the following command wget -qO- http://spatialhadoop.cs.umn.edu/downloads/spatialhadoop-2.3-2.tar.gz | tar --directory $HADOOP_HOME -xvz

Notice that you need to do this for EVERY machine in your cluster, not just the master node. After that, you need to restart the cluster to ensure that the new classes are loaded. You should be good to go from that point.

There are alternative techniques to install SpatialHadoop as you can find on the following page http://spatialhadoop.cs.umn.edu/compile.html

Let me know if this resolves your problem.

aseldawy avatar Jun 03 '15 20:06 aseldawy

solução aseldawy!! obrigado deu certo.

JeffersonEmanuel avatar Aug 30 '15 04:08 JeffersonEmanuel

@aseldawy You say that spatialhadoop-2.3.2 works with Hadoop 2.x, are you sure this covers Hadoop 2.6.0? Having the same issue as OP.

kongshem avatar Feb 29 '16 09:02 kongshem

To run on Hadoop 2.x, you will have to compile it yourself from the source code. Please check the compilation guide at https://github.com/aseldawy/spatialhadoop2/wiki/Compile-SpatialHadoop-from-source

aseldawy avatar Mar 10 '16 03:03 aseldawy

Hey,

I am not able to see the conf, lib folders when i untar the downloaded SpatialHadoop bundle. Please guide me.

NishantBillava avatar Jun 29 '16 22:06 NishantBillava