software-layer icon indicating copy to clipboard operation
software-layer copied to clipboard

GitHub API rate limits hit when using `EESSI-pilot-install-software.sh`

Open ocaisa opened this issue 2 years ago • 1 comments

I was developing for #212 and I hit the rate limits for GitHub after running that script only a couple of times. This is because there are many calls to eb which include --from-pr (and friends).

Using authenticated calls to the API would kick the can down the road for this (and we should be able to do this for the bot), but in general there is probably a better way to check if something is installed rather than calling the full eb command again (which is also quite expensive in terms of time). One approach could be to maintain a list of installed easyconfigs and check if an easyconfig is in the list or not before attempting the eb call.

There has already been some movement towards encapsulating the installation of an easyconfig in that script a bit better and we can see the shape appearing of an appropriate bash function:

EB=$1
target_ec=$2
ec_options=$3
ec_default_options="--robot"
list="/path/to/list/of/installed/easyconfigs/for/arch"

ec_installed=`grep -Fx "$target_ec" "$list" >/dev/null; echo $?`
case $ec_installed in
  0)
    # easyconfig installed
    echo ">> Target $target_ec already installed, skipping..."
    ;;
  1)
    # easyconfig not installed
    echo ">> Installing $target_ec..."
    ok_msg="$target_ec installed!"
    fail_msg="Failed to install $target_ec, woopsie..."
    $EB $target_ec $ec_default_options $ec_options
    check_exit_code $? "${ok_msg}" "${fail_msg}"
    echo $target_ec >> $list
    ;;
  *)
    # some kind of error
    fatal_error "Checking for in $target_ec in $list returned error code $ec_installed"
    ;;
esac

Including some code for a rebuild case should also be straightforward enough.

The easystack format does not (right now) save us from the GitHub API calls, so that wouldn't rescue us here.

ocaisa avatar Jan 12 '23 09:01 ocaisa