How to define my own data type with SpatialHadoop

[gfl94]

Hello, I want to define my own object with SpatialHadoop to run a few experiments, I've got this tutorial from your website, a nice tutorial for beginners. But there are still one thing unclear for me:

When I want to use my own data type, the only way in my mind is that I can write a RectangleId class in package edu.umn.cs.spatialhadoop.core and recompile the whole project then run an experiment, is there any better solution?

[aseldawy]

No. You don't have to write it inside the core package. You can generate a separate JAR out of it and use it in the command line. Let's say you're trying to run a range query using a new datatype com.example.RectangleID. First, you package your class along with any required classes in a JAR file. Then, you use the following command shadoop rangequery -libjars <Created JAR file> shape:com.example.RectangleID Please let me know if you have any further questions

[Elbehery]

Hi,

I have created my custom type as mentioned in the tutorials. I have used the generate operation to generate synthetic data out of the custom shape. However, I got NullPointerException Again. and a warning says Unknown Shape

Below is the command I have used, and I have attached the built jar of the custom type.

shadoop@shadoop-vm:~/spatialhadoop-2.3-hadoop-1.2.1/bin$ ./shadoop generate testcase6.grid mbr:0,0,1000000,1000000 size:100.mb -libjars /home/shadoop/shadoopExternalJars/spatialhadoopdemo-1.0-SNAPSHOT.jar shape:tlabs.tuberlin.tutorials.RectangleID 15/09/09 11:14:59 WARN spatialHadoop.OperationsParams: unknown shape type: 'tlabs.tuberlin.tutorials.RectangleID' 15/09/09 11:15:00 WARN spatialHadoop.OperationsParams: unknown shape type: 'tlabs.tuberlin.tutorials.RectangleID' java.lang.NullPointerException at edu.umn.cs.spatialHadoop.RandomSpatialGenerator.generateMapReduce(RandomSpatialGenerator.java:69) at edu.umn.cs.spatialHadoop.RandomSpatialGenerator.main(RandomSpatialGenerator.java:250) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at edu.umn.cs.spatialHadoop.operations.Main.main(Main.java:101)

I have attached a screenshot of the jar opened, having the fully qualified name.

[aseldawy]

You cannot use a custom datatype with the generate command because SptialHadoop does not know how to randomly generate objects of that type.

[Elbehery]

Yeah, thats why it treats the type as a normal rect object. Got it !!

If I would like to implement such functionality, any hints where to go ?