Add fetcher-default as a plugin

Open balashashanka opened this issue 5 years ago • 4 comments

Currently the fetcher default is in package edu.usc.irds.sparkler.util, this issue is to move this as a plugin.

May 05 '20 10:05 balashashanka

Started working on this will be creating a pull request soon.

May 05 '20 10:05 balashashanka

Thanks. we kept the default implementation in the sparkler-app itself for a few simplifications.

FYI, there are three fetcher plugins already if you want to test plugin loading (without having to develop a new one)https://github.com/USCDataScience/sparkler/tree/master/sparkler-plugins

That being said, all improvements are welcomed. just trying to make it easy for you.

May 05 '20 22:05 thammegowda

Hi I saw this is as one of the the TODO's in the code so thought it would be a good start. But i could work on other things if this plugin is not in priority.

May 06 '20 03:05 balashashanka

You were facing an issue regarding deployment of plugins on sparkler cluster, right?
I suggest resolving that as a start.

May 06 '20 19:05 thammegowda