telemetry-analysis-service icon indicating copy to clipboard operation
telemetry-analysis-service copied to clipboard

Add support for scheduled Zeppelin notebooks

Open maurodoglio opened this issue 7 years ago • 3 comments

To fully embrace Zeppelin as a first class citizen in our data analysis tools, we need to provide ATMO users with the ability to schedule Zeppelin notebooks. Here is a draft list of use cases:

  • As a user, I want to be able to create a new scheduled job provided a Zeppelin notebook.
  • As a user, when I schedule a Zeppelin notebook, I expect the notebook to be rendered in the job detail view.
  • As a user, I expect to see the list of results, logs and runs of a scheduled Zeppelin notebook in the job detail view.

Here is a draft list of tasks required to make it work:

  • [ ] Modify batch.sh to make it use zeppelin executor when dealing with a Zeppelin notebook.
  • [ ] Modify the job detail view to render a Zeppelin notebook either using the zeppelin markdown converter or via a new html/js converter.
  • [ ] Modify the job creation form to also accept Zeppelin notebooks.

Feel free to change this list as needed. Once this list is stable we can open one issue per task.

maurodoglio avatar Jun 21 '17 10:06 maurodoglio

@comloo is going to give us a hand to implement this feature.

maurodoglio avatar Jun 21 '17 10:06 maurodoglio

Wendy's code has been merged, assigning to Rob Hudson who will do some final clean-up before closing.

rafrombrc avatar Aug 30 '17 18:08 rafrombrc

Here's what I consider left before calling this done:

  • Add some extra unit testing around the Zeppelin job provisioners
  • Minor UI cleanup viewing completed notebooks
  • Push out the EMR bootstrap bucket to production

robhudson avatar Aug 30 '17 20:08 robhudson