PowerShell
Common tests should be changed to show detailed error
This has become a problem when AppVeyor changed the way they present the log, only showing the last 2000 log entries. I have opened an issue for this https://github.com/appveyor/ci/issues/2109.
Some of the common tests today is testing "things" in group, for example markdown lint, file contains no new line, etc. A example is when the tests "Should not contain files without a newline at the end" fail with an error, the resulting error is presented in the Tests "pane" like below. It does not show what file actually failed. To find the file that has the problem one need to download the text log and search for the error.
This is very problematic for new users, to find the errors when it's not as easy anymore to scroll down an look for red text that indicate error. Nor is it possible to link to a specific line for reviewers any longer, so having the "test" pane show detailed error would help a lot.
I suggest we change the tests so that each file is it's own It-block, resulting in an error that can easily be read and found where to resolve the problem. This has been down in other tests, like
https://github.com/PowerShell/DscResource.Tests/blob/f9e3dfd3eb9d7b6a9225e2cfba68b79ffe1cab36/Meta.Tests.ps1#L172-L191
Very good point @johlju. Just so I'm clear I understand: is the plan to ensure each of the meta.tests that examine individual files that we change so each is an in an individual IT block? If so, then this makes sense.
each is an in an individual IT block
Exactly. Then it will be an individual row in the "Test pane". This will probably increase the testing time with a few seconds.
We also got https://github.com/appveyor/ci/issues/2109 fixed today with a new "View full log"! So maybe we could hold off with this one for a while.