aztk

aztk copied to clipboard

jafreck

Can't install on WSL because of cryptography dependency

2

Attempting to install 0.7.0 on Windows Subsystem for Linux fails on the `cryptography` dependency, which some Googling suggests isn't supported yet on WSL. ``` x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2...

revodavid

currently, if a bad docker-repo is configured, the cluster will still be provisioned. Failure will only occur once the cluster tries to download the image. At this point the cluster...

jafreck

Spark UI for AZTK jobs

4

Is there a way to ssh into the cluster and look at the Spark UI for the job being run?

shtratos

We should only display this link if history server is (supposed to be) running. We can add this to the cluster definition so on a get/ssh we know the expected...

jafreck

Make plugins Docker containers

3

Use Docker compose to orchestrate the containers Each plugin can either be : * a simple script as it is now(Run on the spark container or on the host) *...

jafreck

Plugins to execute url

1

use url instead of local file for custom script

jiata

jafreck

https://github.com/amplab/SparkNet#installing-sparknet-on-an-existing-spark-cluster Instructions above

jiata

its confusing/annoying when you have to add your storage account keys to the secrets.yaml but are not able to use that storage account for spark jobs. If we automatically propagate...

jiata