cffconvert-github-action icon indicating copy to clipboard operation
cffconvert-github-action copied to clipboard

Test for actual output, not just exit code

Open jspaaks opened this issue 3 years ago • 8 comments

The tests in https://github.com/citation-file-format/cffconvert-github-action/blob/a73af040d6aeb196ca0f696e6fdcc72663c5a96b/.github/workflows/selftest.yml test for exit code. It would be better if we can test for their actual output.

jspaaks avatar Jan 20 '22 12:01 jspaaks

How do we test a solution? Pushing and checking?

abelsiqueira avatar Jan 20 '22 12:01 abelsiqueira

Maybe using https://github.com/nektos/act?

abelsiqueira avatar Jan 20 '22 12:01 abelsiqueira

Not sure what would be a good way to do this, maybe @sverhoeven can help?

jspaaks avatar Jan 20 '22 14:01 jspaaks

You could capture the stdout and compare it with a expected output using https://man7.org/linux/man-pages/man1/comm.1.html

sverhoeven avatar Jan 21 '22 08:01 sverhoeven

We are having issues capturing the stdout.

abelsiqueira avatar Jan 21 '22 08:01 abelsiqueira

The args: passed can't have a > output because the docker cffconvert thinks it is an argument. From what I saw, Faruk and Jurrian tried to get the output from the actions interface, but also could not find a way.

abelsiqueira avatar Jan 21 '22 08:01 abelsiqueira

Might need a composite action.yml or perhaps setting a different entrypoint (to a script that captures the dockerized cffconvert's stdout) could also work:

  • https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-composite-actions
  • https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsentrypoint

jspaaks avatar Jan 25 '22 11:01 jspaaks

Testing @nlescgt

abelsiqueira avatar Jan 25 '22 13:01 abelsiqueira