Test more packages before publishing a new release

[lvjr]

Brief outline of the enhancement

Recently some packages broke after a new l3kernel release had been out. Therefore I think it might be a good improvement to test more packages before publishing a new latex2e (or l3kernel) release.

Mr. Wright replied in https://github.com/lvjr/tabularray/issues/474#issuecomment-1946122548:

We've considered that before but it's not workable: changes in packages would lead to breakage we can't track, etc.

But after doing some experiments I think it is doable. Island of TeX provides weekly updated Docker images for latest full TeX Live. So we can run tests on them with GitHub Actions.

Step 0: Find out all .sty and .cls files in tex/latex folder and compile them with pdflatex on current TeX Live.

% for somename.cls file
\documentclass{somename}
\begin{document}
TEST
\end{document}

% for somename.sty file
\documentclass{article}
\usepackage{somename}
\begin{document}
TEST
\end{document}

Then add every package file which fails the compilation to an ignorelist. (If you want to do more you could also consider tex/xelatex and tex/lualatex folders as well as xelatex and lualatex programs in the future.)

We only need to do Step 0 once. Now every time a new latex release is out, we need to do the following two steps:

Step 1: Do tests similar to Step 0, but we exclude package files in ignorelist and record every success package file in a passlist.

Step 2: Install the new latex release to TeXLive Docker image. And this time we only test files in passlist. If a package file in passlist fails the test, we are almost sure that it is caused by the new latex release.