software-layer
software-layer copied to clipboard
GitHub API rate limits hit when using `EESSI-pilot-install-software.sh`
I was developing for #212 and I hit the rate limits for GitHub after running that script only a couple of times. This is because there are many calls to eb
which include --from-pr
(and friends).
Using authenticated calls to the API would kick the can down the road for this (and we should be able to do this for the bot), but in general there is probably a better way to check if something is installed rather than calling the full eb
command again (which is also quite expensive in terms of time). One approach could be to maintain a list of installed easyconfigs and check if an easyconfig is in the list or not before attempting the eb
call.
There has already been some movement towards encapsulating the installation of an easyconfig in that script a bit better and we can see the shape appearing of an appropriate bash function:
EB=$1
target_ec=$2
ec_options=$3
ec_default_options="--robot"
list="/path/to/list/of/installed/easyconfigs/for/arch"
ec_installed=`grep -Fx "$target_ec" "$list" >/dev/null; echo $?`
case $ec_installed in
0)
# easyconfig installed
echo ">> Target $target_ec already installed, skipping..."
;;
1)
# easyconfig not installed
echo ">> Installing $target_ec..."
ok_msg="$target_ec installed!"
fail_msg="Failed to install $target_ec, woopsie..."
$EB $target_ec $ec_default_options $ec_options
check_exit_code $? "${ok_msg}" "${fail_msg}"
echo $target_ec >> $list
;;
*)
# some kind of error
fatal_error "Checking for in $target_ec in $list returned error code $ec_installed"
;;
esac
Including some code for a rebuild case should also be straightforward enough.
The easystack format does not (right now) save us from the GitHub API calls, so that wouldn't rescue us here.