HTML test improvements

[BreadGenie]

This issue is more of a discussion/brainstorming thread to improve our HTML test suite.

Currently for HTML test, we are comparing a sample HTML report with the generated report. And we probably need to improve it since the tests are very brittle to code changes.

One way to approach it is to check if

• the data such as the product, version and vendor in sections like product_row and
• the HTML structures like table, charts etc are rendered properly (testing data/behaviour/structure instead of code).

We could use beautifulsoup to extract necessary data without the HTML clutter and test it easily.

[terriko]

Reminding @anthonyharrison that he wanted to add some resources here.

I like the beautifulsoup idea, though. it won't solve our javascript "can you click this dropdown and have it show you the right thing" type testing needs, but I think it'd give us a set of base "is the information available in the document" type tests that would be a lot more resilient to change than our current test (which isn't resilient to anything)

[anthonyharrison]

@terriko @BreadGenie A number of possible tools to consider include

• Selenium
• Robot Framework
• Playwright

All provide support for web automation and all support Python. I think it would be worth trying to write the same simple test with each one and then downselect the tool which seems most appropriate for the type of tests which are needed.