Add cblas level 1 routines and corresponding regression tests

[tansongchen]

I'm currently looking on LIT's documentation to learn how to write more comprehensive test, but let's just first put it here.

[tansongchen]

The test passed for higher versions of LLVM but not for lower versions, and I think this is because I generated this .ll file with LLVM 13. @wsmoses Could you maybe tell a little bit on how to generate .ll files that are compatible with lower versions?

[wsmoses]

The test passed for higher versions of LLVM but not for lower versions, and I think this is because I generated this .ll file with LLVM 13. @wsmoses Could you maybe tell a little bit on how to generate .ll files that are compatible with lower versions?

The error is that in lower versions of llvm arguments need actual names, not just %0. Add names to the functions and it should pass?

[tansongchen]

The test passed for higher versions of LLVM but not for lower versions, and I think this is because I generated this .ll file with LLVM 13. @wsmoses Could you maybe tell a little bit on how to generate .ll files that are compatible with lower versions?

The error is that in lower versions of llvm arguments need actual names, not just %0. Add names to the functions and it should pass?

Thanks for the hints. By the way, I am curious about what the life cycle for llvm versions is

[tansongchen]

I uploaded the implementation of level 1 routines and level 2 gemv, but the test system is not working on my computer, I will fix it and add test cases for gemv later

[tansongchen]

Also I noticed that there are some related implementation of gemm at #308 , but both of us are doing this in a bit awkward way. After testing gemv I will try to extract the common portion of many blas routines and do a refactoring

[tansongchen]

@wsmoses All tests have passed. Two fails are due to other test cases

[vchuravy]

@tansongchen you marked all the conversations as resolved, but there is no new commit yet? Maybe you forgotten to push?

[tansongchen]

@tansongchen you marked all the conversations as resolved, but there is no new commit yet? Maybe you forgotten to push?

Just pushed

[wsmoses]

This will likely need to take into account operand bundles as well once that lands, see https://github.com/wsmoses/Enzyme/pull/491

[tansongchen]

@wsmoses I am looking at the operand bundles, and I am wondering why the ValueType for x is Shadow at https://github.com/EnzymeAD/Enzyme/blob/main/enzyme/Enzyme/AdjointGenerator.h#L4777 ? I assume that since memcpy is using the x value this ValueType should be Primal.

[wsmoses]

So the reason is that we’re copying into the shadow

[tansongchen]

I see, I thought Shadow was only used for derivative values... Maybe we should update the description at https://github.com/EnzymeAD/Enzyme/blob/main/enzyme/Enzyme/Utils.h#L258 later

[tansongchen]

Operand bundles are added, and test cases are fixed according to the new convention.

[tansongchen]

@wsmoses Any comments on the latest push?

[tansongchen]

I haven't quite understood what is happening with the Julia nightly CI. It seems that other PRs are also failing on this?

[wsmoses]

Yeah feel free to ignore Julia nightly CI.

[wsmoses]

Modulo the rebase on https://github.com/EnzymeAD/Enzyme/pull/528 and potentially the pass-by-reference handling required this looks good!

Also we're going to want to list the blas routines behaviors in differential use analysis and activity analysis, since in theory we should be able to have stronger information than a generic call fallback.

[vchuravy]

bump! It would be really good to get this done. This is the biggest blocker for Enzyme.jl currently

[tansongchen]

Let's get this merged step-by-step. I am firstly adding swap and copy functions here, and then the remaining level 1 functions, and then level 2, and then level 3. A helper function is implemented to handle the case where we swap a active vector with a constant vector.

[wsmoses]

Reopening to preseve tests which we should add cc @ZuseZ4