Nicholas Chammas
Nicholas Chammas
## Feature request ### Overview If I write to a Delta table, the write operation updates the table's version. There is no way to determine what version was just created...
This is a mystery that someone can take on for fun or for glory. If I change [these](https://github.com/nchammas/flintrock/blob/f662022024603199e25eb4c6f708c176170af1f6/flintrock.py#L601-L602) [two](https://github.com/nchammas/flintrock/blob/f662022024603199e25eb4c6f708c176170af1f6/flintrock.py#L1165-L1166) blocks of code from this: ``` python master_host=master_instance.public_dns_name, slave_hosts=[i.public_dns_name for i...
With the Spark 3.0 transition, there are some version compatibility issues that will likely hit users more often than normal. * Spark is pre-built either against Hadoop 2.6, 2.7, or...
Flintrock supports launching clusters using spot instances. However, when doing so, all the cluster instances are launched as spot instances. Generally, though, you don't want the master to be a...
As requested here: https://github.com/nchammas/flintrock/issues/222#issuecomment-347917936 Related to #199.
Since its creation, Flintrock has sourced Spark releases from [`s3://spark-related-packages`](https://s3.amazonaws.com/spark-related-packages), an S3 bucket hosted by the AMPLab and kept up-to-date by the Apache Spark project. As of Spark 2.2.1, the...
As raised here: https://github.com/nchammas/flintrock/issues/227#issuecomment-348549672 Flintrock depends on some tools being available on the remote hosts it's managing, including: * yum * Python * curl * Bash It would provide a...
Sometimes, especially if you are debugging something, you want to see a little more information on what Flintrock is doing. One type of information I've often found myself wanting to...
If a cluster launch is interrupted before AWS can even return a list of instances, we hit a part of the code where [`cluster_instances` is not defined](https://github.com/nchammas/flintrock/blob/291745ca57566c51bfe7366692e6cb39cb607e72/flintrock/ec2.py#L939). We should protect...
Would it be useful for users if Flintrock offered support for [Apache Zeppelin](https://zeppelin.incubator.apache.org/) as an optional module? I haven't used Zeppelin, but the idea is that Flintrock would install it...