Adding MPI test

[michel2323]

Let me know if you want to keep the trailing whitespaces.

[michel2323]

https://github.com/EnzymeAD/Enzyme.jl/issues/585 @vchuravy This should be merged I guess. Just checking again.

[vchuravy]

Can you rebase once more?

[michel2323]

@vchuravy @wsmoses The MPI test crashes now. With 1.9 we have the finalizer issue, and with 1.8 the logs and Manifest.toml are attached. log.tar.gz

[wsmoses]

Can you open an issue with the error message?

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.43%. Comparing base (90e07c1) to head (f16f50b). Report is 6 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #518       +/-   ##
===========================================
+ Coverage   75.39%   93.43%   +18.03%     
===========================================
  Files          35        7       -28     
  Lines       10653      259    -10394     
===========================================
- Hits         8032      242     -7790     
+ Misses       2621       17     -2604     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[michel2323]

@wsmoses @vchuravy I marked the test here as broken so we can merge the MPI tests and mark it unbroken with the fix. Issue opened https://github.com/EnzymeAD/Enzyme.jl/issues/1138.

[wsmoses]

@michel2323 i rebased this PR with the jll with your fix, if it passes let's merge this!

[wsmoses]

@vchuravy this still fails presumably for the need for: https://github.com/EnzymeAD/Enzyme.jl/pull/669

from

2023-11-20T03:26:35.5348040Z error: /home/runner/.julia/packages/MPI/hhI6i/src/api/generated_api.jl:2151:0: in function preprocess_julia_Isend_50847 {} addrspace(10)* ({} addrspace(10)*, i64, {} addrspace(10)*): Enzyme: failed to deduce type of copy   %65 = call i32 @MPI_Isend(i64 %58, i32 %value_phi4, i32 %61, i32 %52, i32 0, i32 %62, i64 %64) #34 [ "jl_roots"({} addrspace(10)* %2, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 139848936934304 to {}*) to {} addrspace(10)*), {} addrspace(10)* %value_phi5, {} addrspace(10)* %value_phi3) ], !dbg !107

2023-11-20T03:26:35.5120729Z   %value_phi5 = phi {} addrspace(10)* [ null, %pass ], [ addrspacecast ({}* inttoptr (i64 139848936932704 to {}*) to {} addrspace(10)*), %L54 ]: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4979683Z   %59 = bitcast {} addrspace(10)* %value_phi5 to i32 addrspace(10)*, !dbg !123: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4980592Z   %60 = addrspacecast i32 addrspace(10)* %59 to i32 addrspace(11)*, !dbg !123: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4981343Z   %61 = load i32, i32 addrspace(11)* %60, align 4, !dbg !123, !tbaa !45: {}, intvals: {}

While I do agree custom rules are good (and we can redo that API to be a custom global invariant rule, the issue of that MPI.Double being hidden behind a separate julia-specific global int is hindering the analysis here (and also likely would for other libXYZ calls potentially, for example CUDA).

[michel2323]

I get this now on the most recent Enzyme build:

ERROR: LoadError: ERROR: LoadError: Enzyme execution failed.
Enzyme: jl_call calling convention not implemented in aug_forward for   %33 = call nonnull {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_finalizer, {} addrspace(10)* noundef null, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 140298333135136 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull align 8 dereferenceable(16) %newstruct15.i) #29, !dbg !149
Stacktrace:
 [1] finalizer
   @ ./gcutils.jl:87
 [2] Request
   @ /disk/mschanen/julia_depot/packages/MPI/z2owj/src/nonblocking.jl:183

This is the request issue @vchuravy mentioned. How do we proceed with MPI in Julia? Should we start an Enzyme extension for MPI/MPI extension for Enzyme?