telemetry-analysis-service
telemetry-analysis-service copied to clipboard
Add support for scheduled Zeppelin notebooks
To fully embrace Zeppelin as a first class citizen in our data analysis tools, we need to provide ATMO users with the ability to schedule Zeppelin notebooks. Here is a draft list of use cases:
- As a user, I want to be able to create a new scheduled job provided a Zeppelin notebook.
- As a user, when I schedule a Zeppelin notebook, I expect the notebook to be rendered in the job detail view.
- As a user, I expect to see the list of results, logs and runs of a scheduled Zeppelin notebook in the job detail view.
Here is a draft list of tasks required to make it work:
- [ ] Modify
batch.sh
to make it use zeppelin executor when dealing with a Zeppelin notebook. - [ ] Modify the job detail view to render a Zeppelin notebook either using the zeppelin markdown converter or via a new html/js converter.
- [ ] Modify the job creation form to also accept Zeppelin notebooks.
Feel free to change this list as needed. Once this list is stable we can open one issue per task.
@comloo is going to give us a hand to implement this feature.
Wendy's code has been merged, assigning to Rob Hudson who will do some final clean-up before closing.
Here's what I consider left before calling this done:
- Add some extra unit testing around the Zeppelin job provisioners
- Minor UI cleanup viewing completed notebooks
- Push out the EMR bootstrap bucket to production