hardcore-sushi
ERROR_CODE_IO_UNSPECIFIED
When opening files or after a while after opening, ERROR_CODE_IO_UNSPECIFIED appears. But with most files, the app simply crashes when opened. If the storage is shared with other apps via the document provider, it works until the RAM (shared) is overloaded when opening the files or loading thumbnails and then all apps crash.
PS.: If you close all apps before that happens, by closing the volume and other file managers, the shared RAM remains full, even if the files can no longer be accessed. The shared RAM is only emptied when DroidFS is reopened.
PSS.: It's been a long time since I've had an app that crashed as often in my tests as this one. It's a change 😄
PSSS.: I would have liked to send the logcat but it doesn't work in the app either.
Unfortunately I cannot help you without more details. Where is the volume located? (shared storage, external SD card, or hidden volume?) What kind of files are you trying to open? What are their size? Could you share a logcat recorded with another app such as LogFox and explain what's happening with the DroidFS built-in logcat viewer?
How did you configure the "Export method" in the unsafe features page?
I'll try my best to help you with testing. I've used DroidFS on internal storage. It seems to happen with media like videos regardless of format. More often with some files and less often with others but still with most. So it should be easy to reproduce. I've used files between 250MB and 1GB. There is no app called LogFox in the Play Store but Logcat Reader Professional seems to be good. I've figure out that when opening files within DroidFS, the app only crashes in this case if thumbnails are enabled for all files of the appropriate size. The thumbnails are never actually created for all media. The important things from the logcat should be in here: Pastebin
The built-in "logcat viewer" doesn't allow highlighting, copying, the button that looks like you can export something has no function and the line break doesn't work either.
The export method is switched on but I didn't use it in this case.
According to your logcat, the app is just running out of memory. What maximum size did you set for thumbnails?
The export method is switched on but I didn't use it in this case.
No I was asking about the value of the "Export method" option at the very bottom of the unsafe features page.
Yes the built-in logcat viewer is very basic, but the save button should open a system window asking where to save the file. What happens in your case?
LogFox is here: https://f-droid.org/en/packages/com.f0x1d.logfox
I had specified that thumbnails were enabled for all files. In this case up to 1GB. For many years it has been an established standard to cache thumbnails for quick retrieval. This cache must then logically be encrypted. Ideally within(!) the volume in a path such as "/.thumbnail/". And then the problem some thumbnails not loading at all.
The "Export method" is default set to Auto and I left it that way.
The built-in logcat causes DroidFS to freeze, so that scrolling no longer works. As if it were too full...🤔 The built-in logcat seems to collect errors and warnings in particular. Apparently DroidFS is so unstable that the logcat becomes too full, which makes things even worse. Returning from an other app to DroidFS no longer works in this state and only a blank anthracite-colored page is displayed. Maybe DroidFS should only be published as an alpha version.
1GB is very large. It means each file up to 1GB in size will be loaded into memory to extract a thumbnail. I guess this fills the memory and starts to make the app lag and eventually crash by exhausting all the memory. Reducing this limit may fix the issue, but of course a more efficient way should be used to load thumbnails. Unfortunately I don't know any Android library that would allow to do so from partial in-memory files.
The built-in logcat causes DroidFS to freeze
Even when you just started the app and not opened any volumes yet? If so, could you share a logcat (externally recoreded) for this issue please?
This also happens with files that are many times over smaller. 1GB is just the largest ones that I've tested it with.
Even if you've just started the app and haven't opened any volumes yet?
Yes, and you already have the logcat, the problem occurred there too and also direct at the start.
MiXplorer is not open source but does not have the memory overflow problem. Maybe you could ask HootanParsa or decompile it with JADX.
The main problem is that the files are no longer deleted from the memory afterwards, but remain retained after creation. This will also lead to a crash if there are small files. But it crashes no matter which export method I use.
Maybe your Glide version is out of date. Check if you have 4.16 installed.
This also happens with files that are many times over smaller. 1GB is just the largest ones that I've tested it with.
I was talking about thumbnails. Even if you don't open any file, the currently visible files in the explorer get loaded into memory if they are below the configured size in order to extract thumbnails. Therefore, the limit should never be set higher than a few hundred megabytes, depending on your device memory.
you already have the logcat, the problem occurred there too and also direct at the start.
I see no mention of LogcatActivity in your provided logs. Could you force close DroidFS, start recording the logcat with another app, open DroidFS and try to save the built-in logcat, without opening any volumes? If it crashes, please send this new logcat.
I'm able to reproduce the document provider freezing issue. I'll try to fix it.
I was talking about thumbnails.
I also talked about thumbnails. Even if the files are less than 250 MB, DroidFS crashes when thumbnails are enabled.
There are at least three errors behind the problem.
- After successfully generating a thumbnail, the file should disappear from the RAM, but it doesn't.
- If a thumbnail cannot be generated (for reasons that are completely obscure to me and to you), DroidFS crashes when the files are opened.
- Sometimes then also ERROR_CODE_IO_UNSPECIFIED is displayed.
- Thumbnails should be cached according to best developer practices. Encrypted in DroidFS, of course.
Finally got the logcat from DroidFS. Only took a week, as if you had to finish typing it up and send it to me, but here it is: DroidFS_2024-11-07_10_00_09.log
OK, I made some changes that should fix most of the issues you mentioned. Could you test this experimental APK and try to reproduce the bugs? https://upload.disroot.org/r/ko6oK_VN#9JgUm3oPsEot6VRF9cbV5zR2ca9l5hzck9gjSmKlARA=
PGP signature
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
SHA256: 3f64453c6b63ddde244404c2a8a1785858bca06dfc9563abaa81289a243168f4
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQS2Tv6GzuHQVPCCFxGv44Q0SkXhOgUCZzcwGgAKCRCv44Q0SkXh
OvPhAQC231JBxEb5I6d20X4WiVcA/L9Ugghl6j1QGDSLQbEBcQD+J8dl/9Y7hCHf
68HR/XYiSZ84DjTcXR1rYVPjfXBWOw4=
=R2Ks
-----END PGP SIGNATURE-----
After successfully generating a thumbnail, the file should disappear from the RAM, but it doesn't.
What makes you think that? I did some basic experimentations and the files seem to be properly freed from memory. However, you might not see immediate memory usage decrease due to the internal JVM heap management.
Wow, that's really much better. It crashes about 150x less often than before. [sic!] It still happens rarely, but I have the feeling that I have to really try hard to get it to crash.
To show where the problem is, I wanted to export the logcat, but that still doesn't work. To continue using DroidFS, I have to close the app. But if I don't close it and wait a few minutes while I write this text, the AOSP file manager will eventually appear when I switch back to DroidFS and then it works. DroidFS_2024-11-15_13_58_27.log
Now that DroidFS doesn't crash as often, I can see that the thumbnails are completely regenerated after each playback. The repetition doesn't seem to make sense, because it takes so long anyway.
DroidFS also doesn't seem to crash anymore when the volume is exported.
Many files furthermore don't get thumbnails. Actually, they're always the same ones. If I put a file into a volume where this normally works and it is the only one in the volume, it gets a thumbnail, but after playing it is no longer loaded. When I tried to copy this file into a volume very often, I noticed that only one of the copies gets a thumbnail.
PS.: Although only one native library is included, the application is more than twice as large.
According to your logcat, it seems the app crashes simply because it is running out of memory. Changing the export method to disk only, reducing the thumbnail size limit, or disabling thumbnails completely may help.
I wanted to export the logcat, but that still doesn't work.
Even after a fresh app start, without opening any volumes? If so, could you attach an externally recorded logcat for this issue?
I can see that the thumbnails are completely regenerated after each playback. The repetition doesn't seem to make sense, because it takes so long anyway.
Yes it's planned to optimize this: #264.
Many files furthermore don't get thumbnails. Actually, they're always the same ones.
What are theirs sizes and file format? If they are not too large, could you send them to me so I can test on my side?
PS.: Although only one native library is included, the application is more than twice as large.
Yes this is a debug APK (not optimized).
According to your logcat, it seems the app crashes simply because it is running out of memory.
That's weird, I have 8GB RAM + 4GB swap. I haven't tested anything that big yet.
Even after a fresh app start, without opening any volumes? If so, could you attach an externally recorded logcat for this issue?
It worked twice after starting the app. Now it doesn't work at all. Logcat I think that the creating log file in DroidFS shouldn't be larger than one MB and else everything older log must be deleted. Otherwise it takes too long. Alternatively, you could add a note that it might take a few minutes.
If they are not too large, could you send them to me so I can test on my side?
While trying to find concrete examples of files that trigger certain behavior, I noticed a few things. Files for which thumbnails work no longer get any as soon as it are 2 or more files that are over 150MB. Then only one of them gets a thumbnail. All small files work normally. At around 100MB it becomes unstable. Sometimes they load normally in the list/grid, sometimes not. This affects all video containers. After a file has been successfully loaded, you can switch to grid or list and the next thumbnail will be generated. You can repeat this until all of them are loaded. If the files are all 150 MB in size, DroidFS has a lot of tolerance before the app crashes when switching between list and grid. If they are 200 MB, the tolerance becomes exponentially smaller and 1-4 files are enough for DroidFS to crash. The log records crashes between 1-185 MB for 11 files: DroidFS_2024-11-15_17_37_01.log The crashes go up to 259 MB and the thumbnails are loaded for that long. Although the limit is 1GB, the crashes stop when 265 MB is exceeded. The thumbnails are then no longer loaded at all. It is as if DroidFS does not even try with files of this size.
What is particularly noticeable is that DroidFS does not take the 2nd video stream into account. This means that if a specific thumbnail has been set for a video, every file manager, every media player, every other app takes it over, even if it has not been developed for years, regardless of the operating system. Only DroidFS ignores this.
Can you please test with this experimental APK? I modified it so it only loads a maximum of 3 thumbnails simultaneously (completely arbitrarily chosen number) . It seems to fix memory issues for me.
Just an idea, no idea if this helps, but maybe only one file should be processed at a time.
When the DroidFS built-in logcat freezes, does the system show the "App not responding" dialog? If so, could you attach the generated ANR trace from /data/anr/? (requires root access).
Yes maybe the page should only display like the latest 500 lines.
I think the thumbnail memory and video stream/metadata issues could be solved by using a proper thumbnailing library. I just picked Glide because it was already used in the app for pictures but it's not very suited for videos.
There is no ANR. But the suggestion of 500 lines sounds reasonable. Do you think that
Just an idea, no idea if this helps, but maybe only one file should be processed at a time.
could work as a temporary solution?
If the UI becomes unresponsive then an ANR dialog should pop up. Could you maybe share a screencast of this freeze?
Here is a debug APK with the 500 lines hard limit applied on the logcat. When saving, the whole logcat will be exported however.
PGP
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
SHA256: cf8bf7d33f1cedbe207c30be2c192ec6b9d7b4a83cc85747879c40deaf9d60af
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQS2Tv6GzuHQVPCCFxGv44Q0SkXhOgUCZztbAQAKCRCv44Q0SkXh
OpVDAQCziXnRW6ZAu6pTrPF9xSuwiDX96053H+z3PfH8bcPQ2QD+K7DonCZ0mJ3p
0aFmtx+B76sB4WF4KsxZCZoOuNId7As=
=jkcm
-----END PGP SIGNATURE-----
Loading the thumbnails sequentially would not really solve the problem as it would just make the process slower for many small files and would not prevent exhausting the RAM if you have one very big file. The only real solution would be not to load the entire file into memory, which probably implies switching to another thumbnailing library.
There doesn't seem to be any change. If I trigger a few crashes (ERROR_CODE_IO_UNSPECIFIED), the logcat stops working. The logcat can easily handle a crash, but if I trigger it more often, it takes longer with each test until the text becomes scrollable, the back button works again, including the one in the navbar. The text is then displayed but it cannot be moved. I think 500 lines is perhaps too many. Perhaps 100 will work? I triggered about 5 crashes and did a test after each one. After the last one, it takes about 1-2 minutes for DroidFS to respond again. It is interesting that the back button in the navigation bar would normally get stuck if an app freezes. This does not happen here. This suggests that Android does not notice that the logcat is frozen in DroidFS. No button then works, except for home, recent apps and the status bar. I tried the 2nd debug version. DroidFS_2024-11-18_16_41_35.log Do you need more?
500 lines should be handled very fast. I guess the app is simply running out of memory, which let me think that the memory is not freed after a crash. Are you able to reproduce this freezing behavior if you force close the app before opening the built-in logcat?
Even if I force quit the app and immediately call up the logcat, the logcat doesn't work. The problem always occurs when I trigger a lot of crashes in a row, otherwise not.
I think I found the cause of the bug: limiting the number of line is sufficient to handle a large static buffer, but if it's continuously increasing very quickly, it overloads the UI and causes the freeze. This new APK should fix this issue. Could you please try this new version?
https://upload.disroot.org/r/suvFFyGZ#onLtATEXSA8liOtr5PiGtCxleu4DJBRjS+v7BQK0fKQ=
PGP
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
SHA256: cc205e045a16783bda29416859776a76aee1c2a195f92ae25acf46818ad9ecfc
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQS2Tv6GzuHQVPCCFxGv44Q0SkXhOgUCZ0nY2gAKCRCv44Q0SkXh
OvU/AQDvTAC1daAQrIISxIPlLVQ5esFw/K014IkDwKBZTCi6gAEA2LxfCc26FM8N
aVWvn/efIjpL2djhP/scM+jxI/yX6Qk=
=wIa8
-----END PGP SIGNATURE-----
The logcat problem is now solved. To test this I crash DroidFS very often. Every time a thumbnail is loaded, I quickly open a video file and then either ERROR_CODE_IO_UNSPECIFIED is displayed, DroidFS crashes or it is redirected to the main screen. DroidFS_2024-11-29_16_40_32.log This happens whether it is just one file or several. This happens even if the thumbnail is not yet displayed. The phone then generally becomes very slow until I close DroidFS.
OK, could you share a minimal volume that triggers the bug? What are the sizes of the files being thumbnailed? Does it also happens when "Expose open volumes" is disabled?
The crashes occur whether "Expose open volumes" is on or not. It needs a file that is smaller than 265 MB, i.e. one which still loads a thumbnail. It should be as close to this size as possible. The closer it is, the more likely it is that this will happen. At 260 MB, DroidFS almost always crashes if you open the file before or shortly after a thumbnail has loaded. At 250 MB, this happens in 40% of cases, at 240 MB 25%, and at 230 MB 10%. I noticed that the more often it is triggered with smaller files, the less often it happens. Probably because DroidFS gets more memory, because perhaps more background applications have already been killed, because that is probably also due to OutOfMemory.