[feature request] links to GH issues and PRs for failed tests

[samuela]

Hi @dfdx, I'm super psyched to see this project. I'm optimistic that it will significantly raise the bar for AD in Julia.

"Feature request": For the tests that are not passing in https://github.com/dfdx/HQDL.jl/blob/main/REPORT.md it would be nice if there was some way to associate them with GitHub issues, PRs, etc on the relevant other repos. I'm not sure what the best approach is here, since (I'm assuming) REPORT.md is auto-generated. In any case, it seems that we ought to take all of the failing tests and hammer on the bugs until we can get them to go green.

[dfdx]

Hi and sorry for the slow reply - I've spent a couple of days exactly figuring out what's wrong with the tests! As it turns out, most failures are due to errors in tests themselves, not in implementations. I haven't checked all of them yet though.

One way to add links to the autogenerated reports is to provide comments along the tests, e.g. something like this:

@inspect! foo(r(10, 20)) comment="https://github.com/..."

A more advanced option would be to scan known repositories searching for similar error messages, but that would be quite advanced, and we are really at the very beginning yet.

As you said, let's go through the list first, find the real issues and see what would work for them.