c-blosc2
c-blosc2 copied to clipboard
Fix Win32 platform
For some reason, the delta_schunk
test keeps failing quite often for the Win32 platform in GH Actions. This cannot be reproduced in any other platform. valgrind on Ubuntu 18.04 does not offer any clue either:
$ valgrind ./bench/delta_schunk
==10777== Memcheck, a memory error detector
==10777== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==10777== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==10777== Command: ./bench/delta_schunk
==10777==
Blosc version info: 2.0.0.beta.6.dev ($Date:: 2020-04-21 #$)
[Compr] Elapsed time: 27.740 s. Processed data: 1.863 GB (0.067 GB/s)
Compression super-chunk: 2000000000 -> 11978500 (167.0x)
[Decompr] Elapsed time: 17.789 s. Processed data: 1.863 GB (0.105 GB/s)
Decompression successful!
Successful roundtrip!
==10777==
==10777== HEAP SUMMARY:
==10777== in use at exit: 0 bytes in 0 blocks
==10777== total heap usage: 212 allocs, 212 frees, 2,052,250,932 bytes allocated
==10777==
==10777== All heap blocks were freed -- no leaks are possible
==10777==
==10777== For counts of detected and suppressed errors, rerun with: -v
==10777== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
The output is pretty clean :-/
Not sure what's going on here...
Is this still an issue?
Absolutely. When Win32 CI crashes, we re-run the check and it normally passes there. Being so elusive does not help to find a fix. All help is welcome.
I see. I have no direct answer, though a possible mismatch between data types that manifests only when particularly large numbers are randomly hit is a candidate.
Regardless, to help with debugging when it happens, we could modify the tests to produce more output. From my experience in conda-forge/c-blosc2-feedstock#22, I would suggest to eactivate buffering for stdout in the test program so that we get all the output in case of a segfault using setvbuf
(or possibly setbuf
).
Do you have a link to a recently failed example that still has the logs? (It may be worthwhile to preserve such logs by attaching them here.)
Thanks for the feedback. Unfortunately I don't keep a recent sample of failure, but it is a good idea to activate setvbuf
and add a failed log here.