qds-sdk-py
qds-sdk-py copied to clipboard
need macro support for hadoopcmd and other cmds
It looks like macro replacement only works when using the hivecmd. I was trying to do the same with shellcmd and the macros were not replaced as specified in the cmd line. The --macro option is also missing for shellcmd.
qds.py --skip_ssl_cert_check --token=$AUTH_TOKEN --url=https://test.qubole.net/api/ --version=latest shellcmd submit -q "hadoop dfs -ls '$scheme$://$base_bucket$/*/' > level_one.txt;" --macros='[{"scheme": "s3", "base_bucket" : "some-dir-test"}]'