Starlight.jl icon indicating copy to clipboard operation
Starlight.jl copied to clipboard

Published 20 hours ago • verified publisher icon jhigginbotham64

GitHub Action for unit tests

Open jhigginbotham64 opened this issue 2 years ago • 7 comments

Running the test suite should be part of our CI/CD setup.

jhigginbotham64 avatar May 04 '22 19:05 jhigginbotham64

While I though initially that using a premade github action would be trivial, it's not here. There's a big problem with CI/CD of graphical libraries: FD052B5F-512E-441A-A420-27B11FC5A363 they need graphics.

The only option to run tests as they stand currently would be a self-hosted action on a machine that has both an alsa sound device and a display that SDL recognizes.

It is certainly to separate tests for individual internal functions out into headless tests, but the meat and potatoes is going to have to be on a graphics capable computer.

As an extension to that, there could be graphical tests. This could include things like:

  • Does the window pop up
  • Is everything that's supposed to be in the window actually in the window
  • Is this readable
  • Select all images of cats (ok maybe not that one)

It's doable, but difficult. The test suite could get pretty annoying to click through that way too...

Borketh avatar May 05 '22 03:05 Borketh

This is obviously much more difficult than I thought, thanks @AstroFloof for exploring this.

We can consider automating GUI tests with the self-hosted option using something like Squish, but...that really has nothing to do with getting the current test suite to run in CI/CD.

What needs to happen for this particular issue is for Starlight and/or Telescope and/or the tests themselves to be modified to run successfully in a headless environment.

Automated GUI tests can come later.

jhigginbotham64 avatar May 05 '22 03:05 jhigginbotham64

Given the existence of #25 maybe this can run headlessly, but it just doesn't run at all right now.

Borketh avatar May 05 '22 12:05 Borketh

I don't think that issue is related to this one. At least I hope it isn't.

In any case there's no reason why SDL and Vulkan can't run headlessly, they have various features to support that use case, it's just a question of how to make it happen. Hence the increased bounty.

jhigginbotham64 avatar May 05 '22 13:05 jhigginbotham64

I'm taking immediate responsibility for this since it affects the contribution workflow so much. I'm going to make the necessary modifications to Telescope and take care of the GitHub workflows. The rest of you can take it easy wrt this issue.

jhigginbotham64 avatar May 08 '22 15:05 jhigginbotham64

Sounds good! I take it you pay yourself the bounty then? :P

Borketh avatar May 08 '22 19:05 Borketh

Money saved is money earned, so ya pretty much. :)

jhigginbotham64 avatar May 08 '22 23:05 jhigginbotham64