Bus errors on ARM

[satta]

I'm forwarding Debian bug #919778 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919778) which deals with build time tests failing due to an alignment issue in MurmurHash. The mash binary is build correctly but then fails with a bus error when trying to calculate sketches.

See https://buildd.debian.org/status/fetch.php?pkg=mash&arch=armhf&ver=2.1-2&stamp=1545838322&raw=0

    dh_auto_test -a
 	make -j8 test VERBOSE=1
make[1]: Entering directory '/<<PKGBUILDDIR>>'
cd test ; ../mash sketch -o genomes.msh genome1.fna genome2.fna genome3.fna
cd test ; ../mash sketch -r -I reads reads1.fastq reads2.fastq -o reads.msh
Sketching genome1.fna...
Sketching genome2.fna...
Sketching genome3.fna...
Bus error
make[1]: *** [Makefile:94: test/reads.msh] Error 135


Backtrace:

#0  0x00638d2e in MurmurHash3_x64_128 (key=0xf523a009, len=len@entry=21, seed=<optimized out>, out=out@entry=0xf6b2dbd4)
    at src/mash/MurmurHash3.cpp:277
#1  0x00635b4c in getHash (seq=<optimized out>, length=length@entry=21, seed=<optimized out>, use64=true)
    at src/mash/hash.cpp:23
#2  0x00639a50 in addMinHashes (minHashHeap=..., 
    seq=0xf523a008 "AGCTTTTCATTCTGACTGCAACGGGCAATATGTCTCTGTGTGGATTAAAAAAAGAGTGTCTGATAGCAGCTTCTGAACTGGTTACCTGCCGTGAGTAAATTAAAATTTTATTGACTTAGGTCACTAAATACTTTAACCAATATAGGCATAGCGCACAGACAGATAAAAATTACAGAGTACACAACATCCATGAAACGCAT"..., 
    length=<optimized out>, parameters=...) at src/mash/Sketch.cpp:601
#3  0x0063a7d6 in sketchFile (input=0x692268) at src/mash/Sketch.cpp:1264
#4  0x006400a0 in ThreadPool<Sketch::SketchInput, Sketch::SketchOutput>::thread (arg=0xffeea370)
    at src/mash/ThreadPool.hxx:182
#5  0xf6cd2bbe in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
#6  0xf6bc94dc in ?? () from /lib/arm-linux-gnueabihf/libc.so.6

[kloetzl]

This looks related to #106 as a bus error hints at unaligned reads which are problemetic on arm. See the comment in the following code quote.

https://github.com/marbl/Mash/blob/aabd5925e7cfc097a8d89e2d8691ac4af5b95d37/src/mash/MurmurHash3.cpp#L52-L65

[satta]

Ubuntu seems to have a patch that at least should address the alignment issues: https://patches.ubuntu.com/m/mash/mash_2.0-2ubuntu2.patch

However, the tests still fail due to different outputs.

[kloetzl]

The patch doesn't solve the problem within MurmerHash. I started working on a portable shared library which mash and other projects can then link to. https://github.com/kloetzl/libmurmurhash