clojure-hadoop
clojure-hadoop copied to clipboard
Nicer way to pass arguments to jobs
It would be nice if there was a nicer functional-feeling clojurey way to parameterise jobs with some config parameters. At the moment if I understand correctly I need to specify a :map-setup which rebinds some global state based on config arguments specified on the command line.
Would it be possible to avoid the need for this rebinding, or at least to hide it behind a nicer defjob macro along the lines of:
(defn my-map
[ngram-order key value]
;; ...
)
(defjob foo
[config]
{:map (partial my-map (:ngram-order config))
; ... etc ...
})
Realise there are probably inherent limitations here in the way hadoop instantiates these things, but thought the API feedback might be useful :)