Hive-mongo
Hive-mongo copied to clipboard
yc-huang
Error in loading storage handler.org.yong3.hive.mongo.MongoStorageHandler
Hi,
Thanks for the hive plug in. I ran into some issue while following readme to test the installation.
I am getting error in loading the storage handler. It may be a path or some environment variable issue.
I verified : <mongo_hive_home>/src/main/java/org/yong3/hive/mongo/MongoStorageHandler.java exists.
mongo_hive home dir: /home/ben/tools/mongo_hive
here is the full log
[DEV] ben@bi-all:~/tools$ $HIVE_HOME/bin/hive --auxpath /home//ben/tools/mongo_hive/mongo-java-driver-2.6.3.jar,/home/ben/tools/mongo_hive/guava-r06.jar, /home/ben/tools/mongo_hive/hive-mongo-0.0.1-SNAPSHOT.jar Logging initialized using configuration in jar:file:/home/ben/tools/hive-0.8.1-bin/lib/hive-common-0.8.1.jar!/hive-log4j.properties Hive history file=/tmp/ben/hive_job_log_ben_201207211824_1059679816.txt hive> create external table mongo_events(id int, dt int, ip string, event string) stored by "org.yong3.hive.mongo.MongoStorageHandler" with serdeproperties ( "mongo.column.mapping" = "_id,dt,ip,event" ) tblproperties ( "mongo.host" = "bi-fb-app.dev", "mongo.port" = "27017", "mongo.db" = "game", "mongo.collection" = "event_20120703" ); Failed with exception org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org.yong3.hive.mongo.MongoStorageHandler FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
could you help to post what's the error message in your hive log (which is located in /tmp/
thx, YC
YC, Thanks for the reply. I think I missed the step of building all the jar files. Since I don't have a java back ground here are my questions:
- Do I need to build all thrre jar files? or can download it. I am running ubuntu mongo-java-driver-2.6.3.jar, guava-r06.jar, hive-mongo-0.0.1-SNAPSHOT.jar or I can download first two of them? Do you have a documentation on the all the build of each of this?
I see you have some answer for the snapshot: https://github.com/yc-huang/Hive-mongo/issues/3. Is the snapshot.jar is for files which you have created [i.e under mongo_hive/src/main/**...
for guava*jatr, I downloaded guava-r06-site.jar, will that work? or i need to do complete build after downloading all the source files
for mongo java driver, i downloaded the tar files. How do i make jar out of it. mongodb-mongo-java-driver-674c521
- After the building jar is completed, should i be able to do just use your readme doc? do in eed to set any other env variables?
- What are the limitation of existing package. Does it support group by etc.
- Any opinion on mongo-hadoop package. Do you know when 10gen is going to support hive. We are storing good amount of data in mongo, and all we care about is running hive. I suppose it will be faster than the regular hadoop+hive as mongo is faster on map-reduce than hadoop. Do you think so?
thanks a lot for your help
since you asked for hive log, here is it
012-07-23 00:24:33,293 WARN conf.HiveConf (HiveConf.java:
2012-07-23 00:24:38,765 ERROR ql.Driver (SessionState.java:printError(380)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
2012-07-23 00:34:26,314 WARN conf.HiveConf (HiveConf.java:
Sorry for miss your email. Now I include a combo jar under 'target' directory you might try without needing guava/mongo-java-driver jar...