clojure-hadoop icon indicating copy to clipboard operation
clojure-hadoop copied to clipboard

Nicer way to pass arguments to jobs

Open mjwillson opened this issue 12 years ago • 0 comments

It would be nice if there was a nicer functional-feeling clojurey way to parameterise jobs with some config parameters. At the moment if I understand correctly I need to specify a :map-setup which rebinds some global state based on config arguments specified on the command line.

Would it be possible to avoid the need for this rebinding, or at least to hide it behind a nicer defjob macro along the lines of:


(defn my-map
  [ngram-order key value]
  ;; ...
  )

(defjob foo
  [config]
  {:map (partial my-map (:ngram-order config))
   ; ... etc ...
  })

Realise there are probably inherent limitations here in the way hadoop instantiates these things, but thought the API feedback might be useful :)

mjwillson avatar Apr 09 '13 16:04 mjwillson