tree-sitter-haskell icon indicating copy to clipboard operation
tree-sitter-haskell copied to clipboard
verified publisher icon tree-sitter

Memory corruption on aarch64-linux

Open twesterhout opened this issue 6 months ago • 6 comments

Hello,

There seems to be a memory corruption happening when processing the highlights queries (https://github.com/nvim-treesitter/nvim-treesitter/blob/master/queries/haskell/highlights.scm) on ARM Linux.

It's been very tricky to debug the failure because Neovim doesn't actually crash, but just exits without displaying any errors... I did manage to record a screen cast where for a split second nvim displays malloc() mismatching next->prev_size (unsorted). Here's a minimal nvim reproducer config with no plugins:

├── init.lua
├── parser
│   └── haskell.so -> /nix/store/5k1xmynyycfc0qrk9r0fzkhd3w2qfqx2-haskell-grammar-0.0.0+rev=0975ef7/parser
└── queries
    └── haskell
        └── highlights.scm

-- init.lua
vim.api.nvim_create_autocmd("FileType", {
    callback = function()
        pcall(vim.treesitter.start)
    end
})

highlights.scm is from https://github.com/nvim-treesitter/nvim-treesitter/blob/master/queries/haskell/highlights.scm and haskell.so is symlinked to $(nix build --print-out-paths --no-link nixpkgs#vimPlugins.nvim-treesitter.builtGrammars.haskell)/parser.

I'd be happy to provide additional information if necessary, but I'm at a bit of a loss on how to debug this issue.

twesterhout avatar Jun 08 '25 19:06 twesterhout

I run into a memory corruption issue that seems related in a project of mine. The issue reproduces reliably and I could get the below information about it. But I am not familiar enough with debugging memory leaks or tree-sitter in general to be able to tackle the issue.

Below steps reproduce the issue reliably in a Fedora 42 VM running on macOS.

git clone --depth 1 --branch v0.2.0 https://github.com/jha-naman/treetags.git
cd treetags
cargo build --release # issue is seen only in release builds

# adjust upper limit if needed. Using a loop since sometimes memory corruption does not occur for the default concurrency level
for i in {1..100}; do
    if ! target/release/treetags --workers $i; then
        echo "running with $i workers failed."
        valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes --verbose target/release/treetags --workers $i
        break
    fi
done

Valgrind output from the VM is below:

==12523== Memcheck, a memory error detector
==12523== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==12523== Using Valgrind-3.25.1-4441567fbe-20250520 and LibVEX; rerun with -h for copyright info
==12523== Command: target/release/treetags --workers 1
==12523==
--12523-- Valgrind options:
--12523--    --tool=memcheck
--12523--    --leak-check=full
--12523--    --show-leak-kinds=all
--12523--    --track-origins=yes
--12523--    --verbose
--12523-- Contents of /proc/version:
--12523--   Linux version 6.16.3-200.fc42.aarch64 (mockbuild@2f8b30dc89e14172be7c3a1c5f129153) (gcc (GCC) 15.2.1 20250808 (Red Hat 15.2.1-1), GNU ld version 2.44-6.fc42) #1 SMP PREEMPT_DYNAMIC Sat Aug 23 17:42:18 UTC 2025
--12523--
--12523-- Arch and hwcaps: ARM64, LittleEndian, v8-fhm-dpbcvadp-sha3-rdm-atomics-fp16-vfp16
--12523-- Page sizes: currently 4096, max supported 65536
--12523-- Valgrind library directory: /usr/libexec/valgrind
--12523-- Reading syms from /home/naman/play/treetags/target/release/treetags
--12523-- Reading syms from /usr/lib/ld-linux-aarch64.so.1
--12523-- Warning: cross-CU LIMITATION: some inlined fn names
--12523-- might be shown as UnknownInlinedFun
--12523-- Reading syms from /usr/libexec/valgrind/memcheck-arm64-linux
--12523--    object doesn't have a dynamic symbol table
--12523-- Scheduler: using generic scheduler lock implementation.
--12523-- Reading suppressions file: /usr/libexec/valgrind/default.supp
==12523== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-12523-by-naman-on-fedora
==12523== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-12523-by-naman-on-fedora
==12523== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-12523-by-naman-on-fedora
==12523==
==12523== TO CONTROL THIS PROCESS USING vgdb (which you probably
==12523== don't want to do, unless you know exactly what you're doing,
==12523== or are doing some strange experiment):
==12523==   /usr/libexec/valgrind/../../bin/vgdb --pid=12523 ...command...
==12523==
==12523== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==12523==   /path/to/gdb target/release/treetags
==12523== and then give GDB the following command
==12523==   target remote | /usr/libexec/valgrind/../../bin/vgdb --pid=12523
==12523== --pid is optional if only one valgrind process is running
==12523==
==12523== Downloading debug info for /usr/libexec/valgrind/memcheck-arm64-linux...
--12523--   Considering /home/naman/.cache/debuginfod_client/c9f03c9c4306dcdeb25abd9dfd504e192d3ab54b/debuginfo ..
--12523--   .. CRC is valid
==12523== Successfully downloaded debug file for /usr/libexec/valgrind/memcheck-arm64-linux
==12523== Downloading debug info for /home/naman/.cache/debuginfod_client/c9f03c9c4306dcdeb25abd9dfd504e192d3ab54b/debuginfo...
--12523--   Considering /home/naman/.cache/debuginfod_client/c3ad68f641adead08539653dd76f29190f049935/debuginfo ..
--12523--   .. build-id is valid
==12523== Successfully downloaded debug file for /home/naman/.cache/debuginfod_client/c9f03c9c4306dcdeb25abd9dfd504e192d3ab54b/debuginfo
--12523-- REDIR: 0x6829fc0 (ld-linux-aarch64.so.1:strlen) redirected to 0x581d1230 (vgPlain_arm64_linux_REDIR_FOR_strlen)
--12523-- REDIR: 0x6829580 (ld-linux-aarch64.so.1:strcmp) redirected to 0x581d1284 (vgPlain_arm64_linux_REDIR_FOR_strcmp)
--12523-- REDIR: 0x68294c0 (ld-linux-aarch64.so.1:index) redirected to 0x581d1258 (vgPlain_arm64_linux_REDIR_FOR_index)
--12523-- Reading syms from /usr/libexec/valgrind/vgpreload_core-arm64-linux.so
--12523-- Reading syms from /usr/libexec/valgrind/vgpreload_memcheck-arm64-linux.so
--12523-- Reading syms from /usr/lib64/libgcc_s-15-20250808.so.1
--12523-- Reading syms from /usr/lib64/libc.so.6
==12523== Downloading debug info for /usr/lib64/libc.so.6...
--12523--   Considering /home/naman/.cache/debuginfod_client/0877903561a9b846a097f888123ae61b7b665a30/debuginfo ..
--12523--   .. CRC is valid
==12523== Successfully downloaded debug file for /usr/lib64/libc.so.6
--12523-- REDIR: 0x71b4c80 (libc.so.6:memchr) redirected to 0x70507b8 (_vgnU_ifunc_wrapper)
--12523-- REDIR: 0x71b8740 (libc.so.6:strlen) redirected to 0x70507b8 (_vgnU_ifunc_wrapper)
--12523-- REDIR: 0x71b4ee0 (libc.so.6:memcpy) redirected to 0x70507b8 (_vgnU_ifunc_wrapper)
--12523-- REDIR: 0x71b56e0 (libc.so.6:memmove) redirected to 0x70507b8 (_vgnU_ifunc_wrapper)
--12523-- REDIR: 0x71b5880 (libc.so.6:memset) redirected to 0x70507b8 (_vgnU_ifunc_wrapper)
--12523-- REDIR: 0x71b8d00 (libc.so.6:rindex) redirected to 0x708df44 (rindex)
--12523-- REDIR: 0x71b0240 (libc.so.6:malloc) redirected to 0x7085450 (malloc)
==12523== Downloading debug info for /home/naman/play/treetags/target/release/treetags...
==12523== Server query failed: No such file or directory
--12523-- REDIR: 0x71b95c0 (libc.so.6:strstr) redirected to 0x70945ac (strstr)
--12523-- REDIR: 0x71bbc80 (libc.so.6:__GI_memchr) redirected to 0x7090070 (__GI_memchr)
--12523-- REDIR: 0x71b0f84 (libc.so.6:realloc) redirected to 0x708c8ec (realloc)
--12523-- REDIR: 0x71bc200 (libc.so.6:__GI_memcpy) redirected to 0x7090c50 (__GI_memcpy)
--12523-- REDIR: 0x71bd3c0 (libc.so.6:__GI_strlen) redirected to 0x708e908 (__GI_strlen)
--12523-- REDIR: 0x71b0b80 (libc.so.6:free) redirected to 0x7088654 (free)
--12523-- REDIR: 0x71b1968 (libc.so.6:calloc) redirected to 0x708c6b4 (calloc)
--12523-- REDIR: 0x71bd280 (libc.so.6:__strlen_asimd) redirected to 0x708e8c0 (strlen)
--12523-- REDIR: 0x71b4d00 (libc.so.6:bcmp) redirected to 0x7092640 (bcmp)
--12523-- REDIR: 0x71bc340 (libc.so.6:__GI_memmove) redirected to 0x70934a4 (__GI_memmove)
--12523-- REDIR: 0x71bd140 (libc.so.6:__memset_zva64) redirected to 0x7092f04 (memset)
--12523-- REDIR: 0x71b27e0 (libc.so.6:posix_memalign) redirected to 0x708d40c (posix_memalign)
--12523-- REDIR: 0x71b6780 (libc.so.6:strchrnul) redirected to 0x7093c68 (strchrnul)
--12523-- REDIR: 0x71b8924 (libc.so.6:strncmp) redirected to 0x708f324 (strncmp)
--12523-- REDIR: 0x71b6840 (libc.so.6:strcmp) redirected to 0x708fde0 (strcmp)
==12523== Thread 2:
==12523== Invalid write of size 4
==12523==    at 0x40CD96C: advance.part.0 (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40D1593: newline_lookahead (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40D1923: tree_sitter_haskell_external_scanner_scan (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x424EC43: ts_parser__lex (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x424FD53: ts_parser_parse (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x422A343: tree_sitter_tags::TagsContext::generate_tags (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40A1AC3: treetags::tag_processor::TagProcessor::worker (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40C532B: std::sys::backtrace::__rust_begin_short_backtrace (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40ADC2B: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x43603AB: std::sys::pal::unix::thread::Thread::new::thread_start (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x719DFE3: start_thread (pthread_create.c:448)
==12523==    by 0x720901B: thread_start (clone.S:79)
==12523==  Address 0x81af0f0 is 0 bytes after a block of size 32 free'd
==12523==    at 0x708C974: realloc (vg_replace_malloc.c:1801)
==12523==    by 0x40CD933: advance.part.0 (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40D1593: newline_lookahead (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40D1923: tree_sitter_haskell_external_scanner_scan (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x424EC43: ts_parser__lex (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x424FD53: ts_parser_parse (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x422A343: tree_sitter_tags::TagsContext::generate_tags (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40A1AC3: treetags::tag_processor::TagProcessor::worker (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40C532B: std::sys::backtrace::__rust_begin_short_backtrace (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40ADC2B: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x43603AB: std::sys::pal::unix::thread::Thread::new::thread_start (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x719DFE3: start_thread (pthread_create.c:448)
==12523==  Block was alloc'd at
==12523==    at 0x70854D0: malloc (vg_replace_malloc.c:446)
==12523==    by 0x40D164F: tree_sitter_haskell_external_scanner_create (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x425220B: ts_parser_parse (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x422A343: tree_sitter_tags::TagsContext::generate_tags (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40A1AC3: treetags::tag_processor::TagProcessor::worker (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40C532B: std::sys::backtrace::__rust_begin_short_backtrace (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x40ADC2B: core::ops::function::FnOnce::call_once{{vtable.shim}} (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x43603AB: std::sys::pal::unix::thread::Thread::new::thread_start (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x719DFE3: start_thread (pthread_create.c:448)
==12523==    by 0x720901B: thread_start (clone.S:79)
==12523==
==12523==
==12523== HEAP SUMMARY:
==12523==     in use at exit: 456 bytes in 1 blocks
==12523==   total heap usage: 184,478 allocs, 184,477 frees, 516,226,439 bytes allocated
==12523==
==12523== Searching for pointers to 1 not-freed blocks
==12523== Checked 260,632 bytes
==12523==
==12523== Thread 1:
==12523== 456 bytes in 1 blocks are still reachable in loss record 1 of 1
==12523==    at 0x70854D0: malloc (vg_replace_malloc.c:446)
==12523==    by 0x435EDFB: std::sys::pal::unix::stack_overflow::thread_info::set_current_info (in /home/naman/play/treetags/target/release/treetags)
==12523==    by 0x435425F: std::rt::lang_start_internal (in /home/naman/play/treetags/target/release/treetags)

jha-naman avatar Sep 01 '25 14:09 jha-naman

With Helix when I open a .hs file on linux-aarch64 I get this error as well.

sudoer777 avatar Sep 01 '25 21:09 sudoer777

I also have this issue. Some .hs files become unopenable via helix with hx app/Main.hs immediately giving malloc(): mismatching next->prev_size (unsorted)

alexplatz avatar Oct 16 '25 14:10 alexplatz

I have also observed this. It doesn't seem to happen on aarch64-darwin, though.

TravisWhitaker avatar Oct 24 '25 00:10 TravisWhitaker

I encountered the same issue (from Helix) on aarch64-linux. I started attempting a fix here

scherna avatar Nov 03 '25 07:11 scherna

I can now confirm that Helix works when using tree-sitter-haskell built from my commit 😄 I edited my languages.toml to include:

name = "haskell"
source = { git = "https://github.com/scherna/tree-sitter-haskell", rev = "378ba1875f273cd91b29dcb04f32cc88a2842aca" }

Then I deleted my Helix runtime's existing runtime/grammars/haskell.so and runtime/grammars/sources/haskell, ran helix --grammars fetch haskell and helix --grammars build haskell, and success!

scherna avatar Nov 03 '25 07:11 scherna