Testing with meson

[hoheinzollern]

Hi! I picked up on your first comment about testing functionality and I tried to setup a test with meson. Now if you run meson test in the build directory, the first test will execute and the output will be logged to a file. I can see that there's an undocumented test command in the source code, so perhaps if you find this feature interesting you could explain how that works. It would be then possible to automate all tests, since you have quite a large test suite.

[jlaurens]

That is definitely an excellent idea, one of the features I am missing for a looooong time.

However, I am currently reorganizing the repository to use tags and GitHub goodies. So there is now a main branch that has some protections, and I am not ready to merge now.

The testing policy must be revisited because it is not completely ready for automation, as far as it can be of course, because some tests rely on 3rd party tools. Moreover, some expectation mechanism must be designed, probably based on post processing the output file. Brain storming requested...

[hoheinzollern]

Sounds very good! I'll put an issue tracking this pull request, and you're welcome to ping me on that issue once you think it's the right time to add testing. Regarding the expectation mechanism, perhaps you can save the expected output for each test in a file, then check that the tool matches the expected output. We can figure out together how to do that.

[jlaurens]

Some preliminary remarks:

• l3build has a testing mechanism based on log file comparison, here we have .synctex files which are somehow much simpler.
• luatex and pdftex don't output exactly the same file because box dimensions are sometimes a bit different
• the input records of the .synctex file are OS dependent: the path separator...
• the input records of the .synctex file are HD dependent because they are full path
• changing the size of a section in .synctex file is not straightforward

So a custom diff command is needed (but that is not very hard)