termux-app

termux-app copied to clipboard

Accessing files on /sdcard can be more than 200x slower (tested on my Pixel 4a) on Android 11 when accessing many small files. I think a potential fix might be to enable MANAGE_EXTERNAL_STORAGE: https://developer.android.com/training/data-storage/manage-all-files

How much MB was the files? Did you mean "acessing files" as opening a file with a text editor like nano or vim?, or acessing files as copying the files from /sdcard to $HOME?

Could you do:

time cp file $HOME
stat file

How fast would you think it must so?

I think a potential fix might be to enable MANAGE_EXTERNAL_STORAGE

And this potential fix will require much more work than you can think. Termux doesn't support target API higher than 28.

running time rm -rv /sdcard/Minetest/ gave me:

real    5m22.694s
user    0m0.400s
sys     0m2.183s

5min is absurdly long for <100MB and <6000 files!

time rm -rv ~/Minetest/ (an identical copy) gave:

real    0m1.584s
user    0m0.030s
sys     0m0.282s

du /sdcard/Minetest/ -h gave:

96M     /sdcard/Minetest/

find /sdcard/Minetest/ | wc -l gave 5787

I guess a basic summary is Android 11's new sdcard filesystem is trash and we really should figure out how to work around it...

out how to work around it

Use root to manually place files to /data/media/* or place files only on internal storage such as $HOME. No other ways to workaround. You can't turn off FUSE overlay on /sdcard without intruding into system.

I created a 100MB file from /dev/urandom and created several copies at ~/storage/downloads and ~/storage/external-1 My results are not the same:

$ dd bs=1000 count=100000 if=/dev/urandom > raw.bin
100000+0 records in
100000+0 records out
100000000 bytes (100 MB, 95 MiB) copied, 21.3016 s, 4.7 MB/s
$ cp raw.bin ~/storage/downloads
$ cp raw.bin ~/storage/external-1
$ time rm raw.bin -rv
removed 'raw.bin'

real    0m0.125s
user    0m0.010s
sys     0m0.100s
$ time rm storage/downloads/raw.bin -rv
removed 'storage/downloads/raw.bin'

real    0m0.144s
user    0m0.020s
sys     0m0.120s
$ time rm storage/external-1/raw.bin -rv
removed 'storage/external-1/raw.bin'

real    0m0.307s
user    0m0.010s
sys     0m0.280s

Tested on Android 6.0.1.

Accessing the sdcard at /storage/3162-3030 gives me a permission denied error.

The /sdcard in my android is just a symlink to /storage/self/primary

$ ls -la
...
lrwxrwxrwx   1 root   root       21 Jul 12 17:35 sdcard -> /storage/self/primary
...
$ time rm /sdcard/raw.bin -rv
removed '/sdcard/raw.bin'

real    0m0.121s
user    0m0.010s
sys     0m0.100s

Android 11, though as my device doesn't have external sd card slot, such test is omitted:

---

Android 6 doesn't upgrade from "general shared storage" to scoped one, so unlike Android 11, performance of internal storage and shared (~/storage/downloads) will be nearly same.

I think a potential fix might be to enable MANAGE_EXTERNAL_STORAGE

And this potential fix will require much more work than you can think. Termux doesn't support target API higher than 28.

I understand, and I hope that other blocking work is resolved sooner than later. For many reasons. MANAGE_EXTERNAL_STORAGE would, among other things, give filepath access to USB drives. What it might not do, however, is meaningfully speed anything up.

This behaviour can not be fixed since that is Android limitation.