aztk

aztk copied to clipboard

Add environment variable option to cluster.yaml

5

```yaml environment_variables: - name: MY_ENV_VAR value: VALUE - name: MY_ENV_VAR2 value: VALUE2 ```

jafreck

``` azure.common.AzureMissingResourceHttpError: The specified container does not exist.ErrorCode: ContainerNotFound ``` To reproduce: 1. aztk spark cluster create --id {cluster_id} 2. aztk spark cluster delete --force --id {cluster_id} 3. aztk spark...

jafreck

Connection aborted white submitting spark job

3

I am submitting spark job using following command: `aztk spark cluster submit --id spark --name kp1 /Users/kumar.pratik/Documents/codebase//targettest-1.0-SNAPSHOT-jar-with-dependencies.jar --class com.pratik.test.MainTest` I always get following error ``` During handling of the above...

k-pratik

aztk spark init --python flag not working as expected

2

When i run `aztk spark init --python`, the `docker_repo` is not updated to the python docker image. This flag should also enable jupyter and jupyter lab automatically.

jiata

Currently aztk spark init --{toolkit} only sets the `docker_repo`. It should also make more sensible defaults for plugins. And instead of silently doing nothing if an existing cluster.yaml is present,...

jafreck

Add ability to stop submitted Spark applications in SDK

2

In interactive mode you can kill running applications through the master ui, but there is no equivalent in the SDK. This would allow application timeout/retries.

jafreck

The user's entire pubilc ssh key is printed to the console when running `aztk spark cluster add-user`.

jafreck

Currently, we upload the aztk directory to the node. It would be best if we install it and it's dependencies to the node. We would need to refactor setup.py since...

jafreck

How to assign environment variable in aztk cluster

3

Hi I have a specific use case to assign environment variable within the aztk cluster is there any way to assign env variable ?

deepika087

Currently, each cluster has a container with a blob containing it's config. We should move all cluster configs to a storage table like: | cluster_id1 | cluster_config dump1 | |...

jafreck