[tests] upgrade CI harness scripts to be more robust

[tijyojwad]

Our CI scripts currently written in bash, and use a lot of env vars and positional arguments. But this setup is ripe for bugs and errors. It's important to upgrade the scripts to be more robust so our testing system itself doesn't silently break.