Additional e2edev enhancements in support of all in one mgmt hub

[linggao]

The items listed below can be broken down in to different issues.

• [ ] use scripts for most of the targets in the Makefile
• [ ] fix timeouts that can cause it to fail on different machines
• [ ] need a way for the e2edev user to be able to modify the all-in-one script and use the local copy on his machine instead of wget-ing it.
• [ ] create a wrapper script that anyone (including non-core anax developers) can easily run on any VM, repeatedly, that is self-documenting (the usage contains all relevant flags and variables). Right now I use the following script as a poor man's substitute for this, but it really needs to be something more robust that the core anax team supports.

#!/bin/bash
if [[ -z "$1" ]]; then
	echo "Usage: e2edev <exchange-version>"
	exit
fi

echo "Testing exchange-api $1 in e2edev..."
docker pull openhorizon/amd64_exchange-api:$1

echo "Stopping horizon.service..."
sudo systemctl stop horizon.service   # this is only needed if you normally use this machine as a horizon edge node

set -e
echo cd $HOME/src/github.com/open-horizon/anax
cd $HOME/src/github.com/open-horizon/anax

echo "Cleaning up from previous e2edev run..."
make -C test realclean

echo "Pulling latest anax repo updates..."
git pull

echo "Cleaning and then building executables and docker images..."
make clean
make
#make fss   # dave says this might not be needed and travis does not do it

echo "Running tests..."
cd test
make DOCKER_EXCH_TAG=$1
# for quickest possible run use: TEST_VARS="NOLOOP=1 NOCANCEL=1 NOHZNREG=1 NORETRY=1 NOSURFERR=1  NOUPGRADE=1 NOSDO=1 NOPATTERNCHANGE=1 NOSVC_CONFIGSTATE=1 NOCOMPCHECK=1 NOK8S=1 NOKUBE=1 TEST_PATTERNS=sall" make test 
echo "make test TEST_VARS='NOLOOP=1 NOUPGRADE=1 NORETRY=1 NOK8S=1 NOKUBE=1 TEST_PATTERNS=sall' DOCKER_EXCH_TAG=$1"
make test TEST_VARS="NOLOOP=1 NOUPGRADE=1 NORETRY=1 NOK8S=1 NOKUBE=1 TEST_PATTERNS=sall" DOCKER_EXCH_TAG=$1
make stop
echo "make test TEST_VARS='NOLOOP=1 NOUPGRADE=1 NORETRY=1 NOK8S=1 NOKUBE=1' DOCKER_EXCH_TAG=$1"
make test TEST_VARS="NOLOOP=1 NOUPGRADE=1 NORETRY=1 NOK8S=1 NOKUBE=1" DOCKER_EXCH_TAG=$1
set +e

# need to do this even if the test fails, if we want to get back to using it as a node
#echo 'Now run: cd $HOME/src/github.com/open-horizon/anax/test && make realclean && sudo systemctl start horizon.service && cd -'
echo 'Now run: cd $HOME/src/github.com/open-horizon/anax/test && make realclean && cd -'

[MOLOCH-dev]

Could I please get this assigned to me? I'd like to work on all 4 tasks And for task 3, could we make the all-in-one script interactive? To be run as "bash <(wget -qO- link-to-deploy-mgmt-hub.sh)", and set certain variables that can be modified? @dabooz @linggao

[unnati914]

I could work on 4th task 👍

[unnati914]

okay @MOLOCH-dev is already working :) No problem you continue I will take some other issue

[MOLOCH-dev]

@linggao @dabooz I have some potential approaches for Task3 of the above issue. Please go through them and let me know if any of these will be suited, and if so, which will be preferred APPROACH 1 : Adding another script to modify just variables

https://github.com/open-horizon/devops/blob/446e55bab9ebd810941c2940d6a8cfcd7b333eca/mgmt-hub/deploy-mgmt-hub.sh#L54

c)  CONFIG_FILE="$OPTARG"

config_file is the bash script with modifiable variables, it is to be downloaded and modified and will autorun the script Get config_file using curl : local httpCode = $(curl -sS -w “%{response_code}” -o $local_script $url 2>$1 ) Either modify the local variables and run or simply run the script after making it executable The config_file will internally create a file with set variables and run the deploy-mgmt-hub.sh script with the -c tag A flag --ismod ( -m) to set if the script is modified or not

For CI/CD : config_file will be run with --ismod set to 0 (default)

PROS : Can add as many variables to be overridden as we like

CONS : Can only modify existing variables

APPROACH 2 : Use a script like this to wget open horizon, set a flag ismod to 1 or 0 The #!/bin/bash if [ ! -f "/tmp/local-script.sh" ] then wget -O https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh fi For CI/CD : config_file will be run with --ismod set to 0 (default) PROS: The entire script can be modified by the user CONS: There may be issues with users not knowing what exactly is to be modified, or issues if the script isn't set as executable

[dabooz]

@MOLOCH-dev Sorry for the delay. I was temporarily redirected and unable to give attention to open horizon. My idea to solve the problem would be to: a) add a new variable to the command which runs e2edev; make test TEST_VARS="...." HUB_CONFIG=/some/file, where b) /some/file is a file in the user's filesystem that contains the all in one hub config they want to use c) provide instructions in the anax/text README for how to create that file, maybe we tell them how to start with the default config file that e2edev uses. d) when the e2edev makefile is running the test target use the user's config file from HUB_CONFIG instead of the config file that e2edev would use as a default.

[MOLOCH-dev]

@dabooz @linggao Could you please review this PR and elaborate on Task 2 (fix timeouts that can cause it to fail on different machines) with an example of such a timeout?