flambo icon indicating copy to clipboard operation
flambo copied to clipboard

Various SQL, api and context functionality adds

Open plandes opened this issue 7 years ago • 0 comments

This pull request includes:

  • Scala -> Clojure recursive data type conversion
  • SQL array type support for StructType creation
  • Various other small utility functions
  • Context creation/caching.

The last one I might need your help. I broke it out into a separate namespace since it seemed to need its own or I thought you might specifically have a place for it. I also thought you'd end up wanting to refactor it quiet a bit.

The idea here is that you only get one Spark context per JVM, so might as well store it in an atom and keep it around without having to create it over and over. This esp. true since most of the initial configuration (expect the app name, which I'm open for suggestions) can be derived from the JVM environment (i.e. properties etc).

plandes avatar Dec 19 '17 02:12 plandes