maven-hadoop icon indicating copy to clipboard operation
maven-hadoop copied to clipboard

Maven Plugin to submit hadoop jobs

Maven Plugin for Hadoop

http://maven-hadoop.blogspot.com .

Copyright (C) 2010 Karthik Kumar

This is a maven plugin for Hadoop issued under Apache License. This software comes with absolutely NO WARRANTY whatsoever. See LICENSE.txt for more details.

Installation

The plugin should be available in the central repository soon.

com.github.maven-hadoop.plugin maven-hadoop-plugin 0.20.1 /opt/software/hadoop

where , hadoopHome is set to HADOOP_HOME installation directory of the version against which this needs to be compiled against.

Usage / Goals:

Warning: the project is in its pre-infancy serving a very limited purpose at the moment of packing files. More use cases are planned down the pipeline.

  • pack:

Currently a single goal is available called as 'pack' that creates the jar file to be submitted to the hadoop job engine.

The jar contains a directory called ./lib , that contains all the dependent jars in the same. The classes of the current project are also available in the same.

The jar would be created in $basedir/target/hadoop-deploy/${ant.project.name}-hdeploy.jar .

$ mvn hadoop:pack

Once created , this could be submitted to the hadoop jar engine

$ $HADOOP_HOME/bin/hadoop jar $basedir/target/hadoop-deploy/${ant.project.name}-hdeploy.jar job.launching.class