EPIC: Automate the update external test material versions

[smlambert]

We have been doing this manually, changing the tag_version values in set_test_info in common_functions.sh, but should also look at what it would take to automate this.

Related: https://developer.github.com/v3/repos/releases/#get-the-latest-release https://stackoverflow.com/questions/58465057/trigger-a-github-action-when-another-repository-creates-a-new-release

Where possible, we want to automate the update of external test material to latest stable versions. In some cases, where the tagging and releases of external test material is 'not predictable' we will still have to manually manage, but in cases where we can automate we should. Additional caveat: some external test material can be used for certain versions of Java, and we may need to support different tags/releases depending on which JDK_VERSION we are testing.

[sophia-guo]

Tried with https://developer.github.com/v3/repos/releases/#get-the-latest-release looks like somerepos supports releases, some not. For exampley: apache/camel-quarkus, quarkusio/quarkus-quickstarts: not https://github.com/sophia-guo/runaqaTest/runs/1311298479?check_suite_focus=true

AdoptOpenJDK/run-aqa: yes https://github.com/sophia-guo/runaqaTest/runs/1311290502?check_suite_focus=true

Looks like it depends on how the release is created. https://gist.github.com/steinwaywhw/a4cd19cda655b8249d908261a62687f8#gistcomment-2602381

[sophia-guo]

Maybe use tags instead though it can't be assumed that the tags are in chronological order

[sophia-guo]

renaissance is a repo which we can use https://developer.github.com/v3/repos/releases/#get-the-latest-release to get latest info.

https://github.com/AdoptOpenJDK/openjdk-tests/issues/2298#issuecomment-787915946

[sophia-guo]

Github api https://developer.github.com/v3/repos/releases/#get-the-latest-release is not always reliable. It appears that it depends on how the Dev creates their "releases". https://gist.github.com/steinwaywhw/a4cd19cda655b8249d908261a62687f8#gistcomment-2602381

For example we can't get apache/camel-quarkus/ releases info. Same for quarkusio/quarkus-quickstarts.

sophiags-MacBook-Pro:bin sophiag$ curl -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/apache/camel-quarkus/releases"
[

]

Take eclispe openj9 as a workable example:

curl -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/Eclipse/openj9/releases/latest"
{
  "url": "https://api.github.com/repos/eclipse/openj9/releases/36655399",
  "assets_url": "https://api.github.com/repos/eclipse/openj9/releases/36655399/assets",
  "upload_url": "https://uploads.github.com/repos/eclipse/openj9/releases/36655399/assets{?name,label}",
  "html_url": "https://github.com/eclipse/openj9/releases/tag/openj9-0.24.0",
  "id": 36655399,
  "author": {
    "login": "pshipton",
    "id": 18122812,
    "node_id": "MDQ6VXNlcjE4MTIyODEy",
    "avatar_url": "https://avatars.githubusercontent.com/u/18122812?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/pshipton",
    "html_url": "https://github.com/pshipton",
    ...
...
}

Probably go through all external repos and first work with the repos which we can get releases info.

[sophia-guo]

It turns out if you don't explicitly create a release help.github.com/articles/creating-releases the tags are not actually being listed as releases. They just look like the release.

The GitHub Repository “Releases” page is not just Releases, the interface co-mingles Releases (something you create on GitHub) and Tags (a native git feature). A Tag points to a specific commit and a Release, in turn, points to a Tag, but they’re co-mingled on the interface because for a consumer it is usually the Tag that matters for both a Tag and a Release. If we visit the Releases of openjdk/jtreg we can see that there are many Tags, but no Releases, whereas in the Releases of adoptium/run-aqa 1 we can see there are many Releases. The difference between a Release and a Tag on the interface can be identified based on the style of information present, and the information included. A Release has a large heading, and a sub-heading that reads “@username released this n time ago” whereas a Tag simply displays the Tag and the commit it points to.

https://github.community/t/releases-api-sometimes-is-not-reliable/190661

[sophia-guo]

Instead of using github REST api, we can use git command git tag --sort=taggerdate | tail -1. The only drawback is we have to do that with checkouting the repo.

[smlambert]

FYI @SAY-droid427

[SAY-droid427]

Thank you @smlambert ! I will work on this issue this week.

[SAY-droid427]

@llxia @smlambert @sophia-guo May I make a PR with the .yaml file as discussed previously in the meet?