accelerate-llvm
accelerate-llvm copied to clipboard
AccelerateHS
FFI fix.
Description
Hello! New libffi library version doesn't contain argInt function. I've made small fix. It's work for me but I'm not sure about it's correctness in general.
Link to libffi: https://hackage.haskell.org/package/libffi-0.2/docs/Foreign-LibFFI-Types.html
Checklist:
- [x] Check on my local project.
- [ ] Run
accelerate-project tests.
Oh thanks for noticing this!
CInt (32-bit) is not the same as Haskell's Int (might be 32 or 64-bit). Can we change this to use argInt32 or argInt64 depending on the size of Int? (Data.Bits.finiteBitSize will tell you the size. It is possible to do it with Template Haskell to do the branch at compile time, e.g. here). Thanks in advance!
Now it compiles but tests are broken with some internal error and I don't know what to do.
nested data-parallelism
mvm: FAIL (expected) (0.05s)
src/Data/Array/Accelerate/Test/NoFib/Sharing.hs:70:
*** Internal error in package accelerate ***
*** Please submit a bug report at https://github.com/AccelerateHS/accelerate/issues
inconsistent valuation at shared 'Exp' tree (sa=43; env=[4])
Note that this error usually arises due to the presence of nested data
parallelism; when a parallel computation attempts to initiate new parallel
work _which depends on_ a scalar variable given by the first computation.
For example, suppose we wish to sum the columns of a two-dimensional array.
You might think to do this in the following (incorrect) way: by constructing
a vector using 'generate' where at each index we 'slice' out the
corresponding column of the matrix and 'sum' it:
> sum_columns_ndp :: Num a => Acc (Matrix a) -> Acc (Vector a)
> sum_columns_ndp mat =
> let I2 rows cols = shape mat
> in generate (I1 cols)
> (\(I1 col) -> the $ sum (slice mat (lift (Z :. All :. col))))
However, since both 'generate' and 'slice' are data-parallel operators, and
moreover that 'slice' _depends on_ the argument 'col' given to it by the
'generate' function, this operation requires nested parallelism and is thus
not (at this time) permitted. The clue that this definition is invalid is
that in order to create a program which will be accepted by the type checker,
we had to use the function 'the' to retrieve the result of the parallel
'sum', effectively concealing that this is a collective operation in order to
match the type expected by 'generate'.
To solve this particular example, we can make use of the fact that (most)
collective operations in Accelerate are _rank polymorphic_. The 'sum'
operation reduces along the innermost dimension of an array of arbitrary
rank, reducing the dimensionality of the array by one. To reduce the array
column-wise then, we first need to simply 'transpose' the array:
> sum_columns :: Num a => Acc (Matrix a) -> Acc (Vector a)
> sum_columns = sum . transpose
If you feel like this is not the cause of your error, or you would like some
advice locating the problem and perhaps with a workaround, feel free to
submit an issue at the above URL.
CallStack (from HasCallStack):
internalError: Data.Array.Accelerate.Trafo.Sharing:704:56
cvt: Data.Array.Accelerate.Trafo.Sharing:762:48
cvt: Data.Array.Accelerate.Trafo.Sharing:764:55
cvt: Data.Array.Accelerate.Trafo.Sharing:764:46
cvt: Data.Array.Accelerate.Trafo.Sharing:696:69
convertSharingExp: Data.Array.Accelerate.Trafo.Sharing:911:25
convertSharingFun1: Data.Array.Accelerate.Trafo.Sharing:299:17
convertSharingAcc: Data.Array.Accelerate.Trafo.Sharing:293:16
convertSharingAcc: Data.Array.Accelerate.Trafo.Sharing:293:16
convertSharingAcc: Data.Array.Accelerate.Trafo.Sharing:286:13
convertSharingAcc: Data.Array.Accelerate.Trafo.Sharing:283:14
convertSharingAcc: Data.Array.Accelerate.Trafo.Sharing:244:3
convertOpenAcc: Data.Array.Accelerate.Trafo.Sharing:162:35
convertAccWith: Data.Array.Accelerate.Trafo.Sharing:159:14
convertAcc: Data.Array.Accelerate.Test.NoFib.Sharing:69:19
(expected failure)
prelude
map
Int64
DIM0
neg: OK (0.24s)
nofib-llvm-native: <stdout>: commitBuffer: invalid argument (invalid character)
Test suite nofib-llvm-native: FAIL
That error means that something went wrong during sharing recovery. As the error indicates, the cause is often a user expressing nested data parallelism in a way we do not support, but I don't see any way your change would cause this. More likely, the cause is just that sharing recovery is a bit flaky, and it sometimes fails.
Hello @dpvanbalen! Yes, you're right: I checked on latest master and catch the same error.
The sharing recovery error is an expected failure, the real error is in the map tests, whatever test comes after neg (I can't remember what that is). Clearing out the cache (usually ~/.cache/accelerate) might help?
I'm super duper behind on everything so no problem for the delays!
@tmcdonell, unfortunately, it didn't help:( Can we merge it like that or do I need to investigate further?
Why not set the version number of libffi as 0.1 in the. cabal file? I got the same problem.
@wujilingfeng hello! There two arguments to not to do this:
libffi0.1 is pretty old and it's incompatible with other libraries.- Behavior in
libffi0.1 is incorrect in same way that was incorrect my first workaround (it doesn't take into account the different byte-length ofInt).
FWIW as a Hackage trustee I revised existing releases of accelerate-llvm-native with libffi < 0.2 so that users do not run into unbuildable configurations with seemingly valid build plans. This will not help materially however.
