pellepl
Various crashes in current master (commit 1393598)
There are failures during tests of three different types:
- trying to write 25 to 1a at addr 00003633 (as part of writing 512 bytes to addr 00003600)
- FATAL OVERLAP: memcpy from 104541b..104545c to 1045420..1045461
- FATAL write addr too high 01fff00c + 00000002 > 00600000
These can be used as inputs to
build/linux_spiffs_test -f afl_test < {one of the files from the above tar file}
I realize that this may not be relevant after the rewrite, but I'll put it here anyway.
Thanks Phil. I'll definitely look at it as the concepts are the same. Also I don't want to leave 0.3 in a sorry state. So it's much appreciated.
Den 12 okt. 2017 05:08 skrev "Philip Gladstone" [email protected]:
There are failures during tests of three different types:
- trying to write 25 to 1a at addr 00003633 (as part of writing 512 bytes to addr 00003600)
- FATAL OVERLAP: memcpy from 104541b..104545c to 1045420..1045461
- FATAL write addr too high 01fff00c + 00000002 > 00600000
minified.tar.gz https://github.com/pellepl/spiffs/files/1377900/minified.tar.gz
These can be used as inputs to
build/linux_spiffs_test -f afl_test < {one of the files from the above tar file}
I realize that this may not be relevant after the rewrite, but I'll put it here anyway.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pellepl/spiffs/issues/180, or mute the thread https://github.com/notifications/unsubscribe-auth/AE3NV8FZgDsp_17KdplWMhnbB6qr3yEUks5srYLKgaJpZM4P2aCO .
since commit f5e26c4e933189593a71c6b82cda381a7b21e41c (fixes #165) i have memory overwrites. i have not yet tested version 0.4.0.
Hmm, commit f5e26c4 should not affect memory overwrites. Did you bisect down to this? The ones @pjsg reported above seem to have been there since the start. 0.4.0 is still under development on my local machine, so it is not accessible yet.
I was two weeks off - I have not tested any newer commit.
The application is a simple logger app which writes numerous large files.
Row 16 shows an impossible value in the column TimeUsed for the SPIFFS_open call. The time is measured in us.
It happens randomly.
The most interesting aspect is when you display the value in hex format.
When I roll in an older version then it works nicely.
The reason why I updated my SPIFFS app is the following:
The write cache for the app is enabled and the test app just writes a short and constant logmessage. Reading the log file back failed.
In my logging I found the situation where SPIFFS writes the internal cache to the flash and in the very next write call the cache is written once again after writing only 21 bytes.
I am using an S25FL164K flash chip with 4kb erasesize. When using a 64kb erasesize the unexpected cache write does not happen.
Is there a test which verifies the internal caching mechanism?
Ok, thanks for the info. It is very weird. No, there are no tests for the internal caching mechanisms I think. Well perhaps some, but these are very basic. But you're sure that before commit f5e26c4 it works?
hmmm, I'm getting older ... For shure commit 7f31842b837445004e62bd92d4d6ee4532eec194 hasn't the internal 'overwrite' bug. I updated inbetween ... but I'm facing a dangling pointer when I did so ...
I know the feeling 😉 Eugh, that commit is almost a year old. Need to narrow it down. What's the dangling pointer problem?
Den 16 okt. 2017 11:48 skrev "robert-b" [email protected]:
hmmm, I'm getting older ... For shure commit 7f31842 https://github.com/pellepl/spiffs/commit/7f31842b837445004e62bd92d4d6ee4532eec194 hasn't the internal 'overwrite' bug. I updated inbetween ... but I'm facing a dangling pointer when I did so ...
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pellepl/spiffs/issues/180#issuecomment-336836706, or mute the thread https://github.com/notifications/unsubscribe-auth/AE3NVwR5Ugx1sJiGRUaBsBoY6mqZXx8Bks5ssyZfgaJpZM4P2aCO .
Yes, the commit is old. I am re-running a test - when finished I can say more. The dangling pointer is somewhere between my ears. I can't remember when I updated SPIFFS ...
lol got it. Appreciate the effort.
Den 16 okt. 2017 13:51 skrev "robert-b" [email protected]:
Yes, the commit is old. I am re-running a test - when finished I can say more. The dangling pointer is somewhere between my ears. I can't remember when I updated SPIFFS ...
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pellepl/spiffs/issues/180#issuecomment-336863668, or mute the thread https://github.com/notifications/unsubscribe-auth/AE3NVxdySdzlRsNJJX_UNVqblIrO6cwiks5ss0M9gaJpZM4P2aCO .
I went back to the SPIFFSversion from the 4.th of august 2017. The 'overwrite' happens during the SPIFFS_open call only. Need a few more Tests - also with the latest version.
I went further back - version v0.3.6. The same problem... Need some ideas.
@robert-b will look into it, but got busy with other commitments right now, sorry.
I did run the linux_spiffs_test using valgrind - the valgrind output is not so amusing. Can you do the same?
==10301== LEAK SUMMARY: ==10301== definitely lost: 0 bytes in 0 blocks ==10301== indirectly lost: 0 bytes in 0 blocks ==10301== possibly lost: 0 bytes in 0 blocks ==10301== still reachable: 6,320,232 bytes in 86 blocks ==10301== suppressed: 0 bytes in 0 blocks ==10301== Reachable blocks (those to which a pointer was found) are not shown. ==10301== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==10301== ==10301== ERROR SUMMARY: 24 errors from 2 contexts (suppressed: 0 from 0) ==10301== ==10301== 9 errors in context 1 of 2: ==10301== Conditional jump or move depends on uninitialised value(s) ==10301== at 0x41220B: _write (test_spiffs.c:208) ==10301== by 0x40E993: spiffs_phys_wr (spiffs_cache.c:215) ==10301== by 0x404A62: spiffs_object_append (spiffs_nucleus.c:1263) ==10301== by 0x40B141: spiffs_hydro_write (spiffs_hydrogen.c:450) ==10301== by 0x40B809: SPIFFS_write (spiffs_hydrogen.c:587) ==10301== by 0x414039: test_create_and_write_file (test_spiffs.c:779) ==10301== by 0x4166C3: __test_index_cons1 (test_check.c:262) ==10301== by 0x428D5E: run_tests (testrunner.c:192) ==10301== by 0x411D08: main (main.c:9) ==10301== Uninitialised value was created by a stack allocation ==10301== at 0x403C5B: spiffs_object_create (spiffs_nucleus.c:918) ==10301== ==10301== ==10301== 15 errors in context 2 of 2: ==10301== Conditional jump or move depends on uninitialised value(s) ==10301== at 0x41220B: _write (test_spiffs.c:208) ==10301== by 0x40E993: spiffs_phys_wr (spiffs_cache.c:215) ==10301== by 0x405622: spiffs_object_append (spiffs_nucleus.c:1436) ==10301== by 0x40B141: spiffs_hydro_write (spiffs_hydrogen.c:450) ==10301== by 0x40B809: SPIFFS_write (spiffs_hydrogen.c:587) ==10301== by 0x414039: test_create_and_write_file (test_spiffs.c:779) ==10301== by 0x415A9D: __test_lu_check1 (test_check.c:58) ==10301== by 0x428D5E: run_tests (testrunner.c:192) ==10301== by 0x411D08: main (main.c:9) ==10301== Uninitialised value was created by a stack allocation ==10301== at 0x403C5B: spiffs_object_create (spiffs_nucleus.c:918) ==10301== ==10301== ERROR SUMMARY: 24 errors from 2 contexts (suppressed: 0 from 0)
Valgrind reports a problem in the test code: TEST 2/89 : running test lu_check1 create and write file check: LOOKUP FIX INDEX 0001:0001 check: LOOKUP DELETE PAGE 0006 .. ok free blocks : 15 of 16 pages allocated : 4 pages deleted : 2 gc runs : 0 cache hits : 53 (sum 53) cache misses : 391 (sum 391) cache utiliz : 0.119369 RD: 138552 reads 778809 bytes 5 avg bytes/read WR: 24 writes 1599 bytes 66 avg bytes/write fs consistency check output begin fs consistency check output end locks : 0 TEST 3/89 : running test page_cons1 create and write file check: PAGE FIX INDEX 8001:0000 check: PAGE FIX INDEX 8001:0000 .. ok free blocks : 15 of 16 pages allocated : 4 pages deleted : 2 gc runs : 0 cache hits : 97 (sum 97) cache misses : 332 (sum 332) cache utiliz : 0.22==25336== Conditional jump or move depends on uninitialised value(s) ==25336== at 0x41220B: _write (test_spiffs.c:208) ==25336== by 0x40E993: spiffs_phys_wr (spiffs_cache.c:215) ==25336== by 0x404A62: spiffs_object_append (spiffs_nucleus.c:1263) ==25336== by 0x40B141: spiffs_hydro_write (spiffs_hydrogen.c:450) ==25336== by 0x40B809: SPIFFS_write (spiffs_hydrogen.c:587) ==25336== by 0x41402A: test_create_and_write_file (test_spiffs.c:779) ==25336== by 0x4166B4: __test_index_cons1 (test_check.c:262) ==25336== by 0x428D4F: run_tests (testrunner.c:192) ==25336== by 0x411D08: main (main.c:9) ==25336== 6107 RD: 138471 reads 764639 bytes 5 avg bytes/read WR: 20 writes 1599 bytes 79 avg bytes/write fs consistency check output begin
I am trying to understand - help is appreciated.
when the memory layout changes valgrind reports other locations ... when running the test under gdb i'm getting: ran 89 tests Test report, 89 tests 85 succeeded 4 failed write_big_files_chunks_page lseek_modification_append_multi long_run_config_many_medium long_run 0 stopped
FAILED
@pjsg I created a pull request. The function "spiffs_create_object" did rely on unitialized memory. My problem seems to be solved - need a longer test run to be sure.