aztk
aztk copied to clipboard
Azure
AZTK powered by Azure Batch: On-demand, Dockerized, Spark Jobs on Azure
currently, when using `--verbose` on the CLI, `log.print` will prefix the first line of output and add new lines. It should have a custom formatter that just prints without modifications.
Hello @jafreck @timotheeguerin Right now AZTK in Spark SDK when [`aztk.spark.client.Client.submit()`](https://github.com/Azure/aztk/blob/828162ef10a80e8b9da8c1345bd12c5235bc1e1f/aztk/spark/client/client.py#L67) is called, it assumes that `ApplicationConfiguration` contains paths to local files in `jars` and `files` fields. In our case...
Related to #679 Currently, if `scheduling_target` is set to `Master`, logs are not available until the completion of Spark application execution. If `scheduling_target` is set to `Any`, then logs can...
As mentioned here https://github.com/Azure/aztk/pull/278, logs are updated to azure blog storage only after app is completed. I noticed that even after stopping an app, the job finished in a completed...
Fix #589 To do: - [x] mount location - [x] expose format type
Job and cluster implementations should be aligned to ease transition between the two modes for users. This will also decrease code complexity/duplication and facilitate support. Currently a cluster is a:...
Paths that are created on the client, but evaluated on the node need to be in linux style. That is currently done ad-hoc and leads to errors with double slashes...
- [ ] Create user with password - [ ] Create user with ssh key - [ ] Generate user on cluster - [ ] Generate user on node
Hello, I'm trying to install aztk, but when I try to create a Cluster, it throws the following error:  Any suggestions on how to install Aztk without issue? The...
