qunitjs.com icon indicating copy to clipboard operation
qunitjs.com copied to clipboard

Document html ID of test element as public API

Open asturur opened this issue 7 years ago • 14 comments

Tell us about your runtime:

  • QUnit version: 2.6.1
  • What environment are you running QUnit in?: Browser

Hi, i started a topic in the gitter chat and i was suggested to come here for a feature proposal. I would like to have a way from the qunit tests, to modify the HTML output of the qunit html reporter.

I m testing canvas differences with one of the libraries available, in case of failure i would like to display the 3 images ( actual, expected, diff ) in the html reporter.

I do understand this is a narrow use case, while maybe an api to define plugins to make this modification possible could be a more generic solution.

What do you think? If you think the idea is nice i would start to study the library to work on it and open a PR with a first draft.

asturur avatar Jun 26 '18 22:06 asturur

If i had to do a quick job i would: take this empty line here: https://github.com/qunitjs/qunit/blob/master/reporter/html.js#L837

add there details.callback(message); Where message is the html that is going to be appended, and write a function that can add my HTML there.

Then i would search the clean way to decorate all the output details with a reference to this function.

What do you think?

asturur avatar Jun 26 '18 22:06 asturur

I don't think an API like this would see much use outside of the specific use case you've presented. That said, you should be able to achieve the same results by registering an additional QUnit.log callback that modifies the output immediately after it is created.

While the markup the reporter generates isn't explicitly considered part of our public API, it is unlikely to change/break any time in the near future, so it should be safe.

trentmwillis avatar Jun 28 '18 03:06 trentmwillis

My solution also would not work. Does Log let me do something with the Html node, or just strings? i need to append pre painted canvas, html markups are not enough, and large dataurls may be a waste of cpu time

asturur avatar Jun 28 '18 06:06 asturur

Sorry for not coming back, had lot of things to do. I would like to make a proposal pr anyway, at least for other people to eventually see how it can be done.

Are you completely sure this is something you do not want?

asturur avatar Jul 12 '18 13:07 asturur

While this is not specified in the docs: https://github.com/qunitjs/qunit/blob/master/src/test.js#L328

can i count this as a part of the details in the API?

asturur avatar Sep 01 '18 08:09 asturur

So with some wild code i could do what i was looking for without modifying the lib or the need for a new api

image

The problem is that the API does not allow me to create a PER TEST callback. i can create a testDone callback but i can't remove it, i can't pass parameters in it at all, so it means i have to rely on external variables.

This is bad.

If we could have a per test callback, or a way to remove a callback, that means i could define a scoped callback from inside the test and remove it after the test is done, and produce a new one in the next test.

asturur avatar Sep 01 '18 08:09 asturur

Indeed what i did work for 1 module, in the next module the second callback stack with the previous and everything break.

asturur avatar Sep 01 '18 08:09 asturur

Would a method to deregister a testDone callback be accepted as a PR?

asturur avatar Sep 01 '18 08:09 asturur

See also https://github.com/Krinkle/node-colorfactory, which uses the DOM API (select by ID) to visualise additional information from a QUnit test onto the HTML report.

Code at /test/testinit.js and /test/testinit.css

Example:

screen shot 2019-01-03 at 12 37 34

Krinkle avatar Jan 03 '19 20:01 Krinkle

i did something similar. unsure if is worth creating a plugin. I did not need to modify any qunit file, but i could plug it from outside.

asturur avatar Jan 03 '19 20:01 asturur

http://fabricjs.com/test/visual/?coverage

not the best visual, but working

asturur avatar Jan 03 '19 20:01 asturur

Yeah, that uses the same interface (the HTML ID qunit-test-output-${testId}).

I think that's fairly minimal and is good enough for this use case. It might change a little bit over time (e.g. the styling), so between QUnit upgrades minor changes may be required, but that seems okay?

Krinkle avatar Jan 03 '19 20:01 Krinkle

Qunit as a test runner has a unique feature of running a bit everywhere. Visual testing would be a nice addition and an official single api to handle dom modifications would be fair too.

Would be enough to consider that htmlID change as a breaking change with a major bump.

Just recognize it as a feature.

asturur avatar Jan 03 '19 21:01 asturur

Yeah, I think the theme and styling changes should not be considered a breaking change (esp. because you can use any theme).

But, the entry point for the DOM (the HTML ID) could be something we can agree to only break with semver-major.

Krinkle avatar Jan 03 '19 21:01 Krinkle