SPIFFS operation becomes faulty when reaching around 80% of usage.

[piotrva]

Describe the bug As observed in #2700 (and 825dea27a0b1631b1325a86f9c0bc110b440f907) when SPIFFS usage reaches about 80% the operation of the memory become erroneous

To Reproduce Steps to reproduce the behavior:

1. mem spiffs wipe
2. script run init_rdv4
3. upload 9 files of 16KB into a memory
4. trying to upload next 16KB file fails (errno -10001), the file size after the upload is not complete

Expected behavior It shall be possible to use most of the SPIFFS memory without such an error.

Screenshots

Desktop (please complete the following information):

• OS: Windows 10 x64
• hw version

[usb] pm3 --> hw version

[ Proxmark3 RFID instrument ]

[ Client ]
 Iceman/extend-spiffs-partition-last-page/e3486...-suspect 2024-12-29 22:21:23 1fa67a415
 compiled with............. MinGW-w64 13.2.0
 platform.................. Windows (64b) / x86_64
 Readline support.......... present
 QT GUI support............ present
 native BT support......... absent
 Python script support..... present ( 3.11.5 )
 Python SWIG support....... present
 Lua script support........ present ( 5.4.6 )
 Lua SWIG support.......... present

[ Proxmark3 ]
 device.................... RDV4
 firmware.................. RDV4
 external flash............ present
 smartcard reader.......... present
 FPC USART for BT add-on... absent

[ ARM ]
 bootrom: Iceman/extend-spiffs-partition-last-page/e3486...-suspect 2024-12-29 22:21:13 1fa67a415
      os: Iceman/extend-spiffs-partition-last-page/e3486...-suspect 2024-12-29 22:21:17 1fa67a415
 compiled with GCC 12.2.0

[ FPGA ]
fpga_pm3_hf.ncd image 2s30vq100 2024-02-03 15:12:20
fpga_pm3_lf.ncd image 2s30vq100 2024-02-03 15:12:10
fpga_pm3_felica.ncd image 2s30vq100 2024-02-03 15:12:41
fpga_pm3_hf_15.ncd image 2s30vq100 2024-02-03 15:12:31

[ Hardware ]
 --= uC: AT91SAM7S512 Rev A
 --= Embedded Processor: ARM7TDMI
 --= Internal SRAM size: 64K bytes
 --= Architecture identifier: AT91SAM7Sxx Series
 --= Embedded flash memory 512K bytes ( 73% used )

• hw status

[usb] pm3 --> hw status
[#] Memory
[#]   BigBuf_size............. 39256
[#]   Available memory........ 39256
[#] Tracing
[#]   tracing ................ 1
[#]   traceLen ............... 0
[#] Current FPGA image
[#]   mode.................... fpga_pm3_lf.ncd image 2s30vq100 2024-02-03 15:12:10
[#] Flash memory
[#]   Baudrate................ 24 MHz
[#]   Init.................... ok
[#]   Mfr ID / Dev ID......... EF / 11
[#]   JEDEC Mfr ID / Dev ID... EF / 3012
[#]   Memory size............. 256 kB (4 pages * 64k)
[#]   Unique ID (be).......... 0x378F9A53175462D5
[#] Smart card module (ISO 7816)
[#]   version................. v4.42 ( ok )
[#] LF Sampling config
[#]   [q] divisor............. 95 ( 125.00 kHz )
[#]   [b] bits per sample..... 8
[#]   [d] decimation.......... 1
[#]   [a] averaging........... yes
[#]   [t] trigger threshold... 0
[#]   [s] samples to skip..... 0
[#]
[#] LF T55XX config
[#]            [r]               [a]   [b]   [c]   [d]   [e]   [f]   [g]
[#]            mode            |start|write|write|write| read|write|write
[#]                            | gap | gap |  0  |  1  | gap |  2  |  3
[#] ---------------------------+-----+-----+-----+-----+-----+-----+------
[#] fixed bit length (default) |  29 |  17 |  15 |  47 |  15 | n/a | n/a |
[#]     long leading reference |  29 |  17 |  18 |  50 |  15 | n/a | n/a |
[#]               leading zero |  29 |  17 |  18 |  40 |  15 | n/a | n/a |
[#]    1 of 4 coding reference |  29 |  17 |  15 |  31 |  15 |  47 |  63 |
[#]
[#] HF 14a config
[#]   [a] Anticol override.... std    ( follow standard )
[#]   [b] BCC override........ std    ( follow standard )
[#]   [2] CL2 override........ std    ( follow standard )
[#]   [3] CL3 override........ std    ( follow standard )
[#]   [r] RATS override....... std    ( follow standard )
[#] Transfer Speed
[#]   Sending packets to client...
[#]   Time elapsed................... 500ms
[#]   Bytes transferred.............. 250880
[#]   Transfer Speed PM3 -> Client... 501760 bytes/s
[#] Various
[#]   Max stack usage......... 3520 / 8480 bytes
[#]   Debug log level......... 1 ( error )
[#]   ToSendMax............... -1
[#]   ToSend BUFFERSIZE....... 2308
[#]   Slow clock.............. 31484 Hz
[#] Installed StandAlone Mode
[#]   LF HID26 standalone - aka SamyRun (Samy Kamkar)
[#] Flash memory dictionary loaded
[#]   Mifare.................. 2311 keys (spiffs: dict_mf.bin)
[#]   T55xx................... 125 keys (spiffs: dict_t55xx.bin)
[#]   iClass.................. 29 keys (spiffs: dict_iclass.bin)
[#]

• hw tune

not related to the issue

[iceman1001]

Most likely the spiffs implementation like to work with full pages.

[piotrva]

Well, the same behavior was for previous implementation with full pages...

[iceman1001]

Was it? Didn't know about that. Is there a update to the spiffs library?

[piotrva]

I observed same behaviour for commit https://github.com/RfidResearchGroup/proxmark3/commit/825dea27a0b1631b1325a86f9c0bc110b440f907 - so preceding modifications to SPIFFS setup. It would be nice if other users can confirm or deny this find. And I see that spiffs has not been updated in recent 9-10 years...

[hornet-hacker]

When attempting to upload a file to SPIFFS I encounter the error above when attempting to exceed 75% (3 of 4 pages) on my RDV4.

[hornet-hacker]

Recreating the bug on a pm3 X with 2mb spi, the bug begins at 79% capacity

[piotrva]

@hornet-hacker thanks for the feedback! Which commit you used for testing?

[hornet-hacker]

@piotrva client version in logs is ga53187071, so I assume that is related to the commit... Edit: looking at the commit list it appears I was on a531870

[piotrva]

@hornet-hacker this is after my changes, so could you please make a test on commit https://github.com/RfidResearchGroup/proxmark3/commit/825dea27a0b1631b1325a86f9c0bc110b440f907 please?

[hornet-hacker]

@piotrva Issue exists on 825dea2

[piotrva]

Testing at: https://github.com/RfidResearchGroup/proxmark3/commit/7082302a02aef0473ab8d931eb3a8cc55353faaa Issue stil present:

[iceman1001]

When attempting to upload a file to SPIFFS I encounter the error above when attempting to exceed 75% (3 of 4 pages) on my RDV4.

That was likely before onRDV4 page 3 (0-index) was reserved for dictionaries and settings. We moved away from that some months ago and now use files instead.

Which user-scenarios does this happen?!? I mean you guys seem to have megabyte spiffs....

[iceman1001]

Ping