hotsub
hotsub copied to clipboard
Command line tool to run batch jobs concurrently with ETL framework on AWS or other cloud computing resources
# positive ``` % docker-machine -v docker-machine version 0.16.0, build 702c267f ```
# Because - Making `runner.sh` for each workflow is redundant - Validation task files is redundant - Checking if the shell script satisfy required params is difficult # I want...
so that user can delete instances without anxiety
1. Machine Name Conflicts - To avoid collision of machine name so that another terminal can hit the same workflow 2. Machine Identifications - So that the machines which are...
- dep?
# As Is Because `platform` package requires `$HOME/.aws/credentials` file [here](https://github.com/otiai10/awsub/blob/master/platform/amazon.go#L37), `$HOME/.aws/credentials` must be located even if the environment is granted instance profile by which it can use `awsub`. # To...
# AS IS `--inputs` and `--input-recursive` must be either of `s3://...` or `gs://...` # TO BE For example, `http://raw.github.com/foo/bar.txt` should be allowed
``` upload failed: tmp/test180204/star/Hela_wt_dox-/Hela_wt_dox-.Chimeric.out.sam to s3://awsub-test-friend1ws2/test180204/star/Hela_wt_dox-/Hela_wt_dox-.Chimeric.out.sam An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist [star-alignment-task00] [FINALIZE] Successfully uploaded: s3://awsub-test-friend1ws2/test180204/star/Hela_wt_dox- ``` I think we...