autoware icon indicating copy to clipboard operation
autoware copied to clipboard

Report results of the weekly scenario tests publicly as dashboard

Open mitsudome-r opened this issue 1 month ago • 0 comments

Description

We should have some public notification about the results of scenario tests without the need of logging in to TIER IV's evaluator console.

Background

Currently, we are doing weekly scenario tests within ODD WG using TIER IV's evaluator functions, and the results are visible under TIER IV's evaluator console. However, you need to create an account and ask admins add give them access to view the results. We should make the result of the weekly scenario tests publicly visible and reported to Autoware GitHub so that developers could know any regression quicker.

Possible Approach

There is WebAutoCLI which can retrieve the results from command line. We can have AWS host which use these commands that retrieves the result of scenario tests periodically and publish the result as dashboard. We could also create an issue in Autoware GitHub for failing issue

Image

mitsudome-r avatar May 15 '24 22:05 mitsudome-r