infinit icon indicating copy to clipboard operation
infinit copied to clipboard

very bad write performances

Open Julio-Guerra opened this issue 6 years ago • 4 comments

I went through the getting started documentation, both on macOS and linux. I end up with the same problem on both: unusable for write-intensive operations.

I reduced the case to a subset of the "get started" doc: one user, one volume, one network (replication = 2), one silo, and everything locally:

I then use any write-intensive program. My favorite simple stress test is using Keynote on a packaged presentation (i.e. it's a package of files instead of a single monolithic one) which modifies a lot of files as soon as you move a single byte in the presentation. In my case, editing a keynote presentation stored in a volume is almost impossible since. Same thing on linux untaring gcc sources (10 minutes for 1.8 GiB) or compiling then (stopped before the end). It's always ~3 MiB/second.

I can provide detailed logs if required.

Julio-Guerra avatar Jul 29 '17 22:07 Julio-Guerra

What is your volume block_size?

tarlano avatar Aug 07 '17 17:08 tarlano

I haven't set it (I'm just following your getting started commands), so the default value, I guess.

Julio-Guerra avatar Aug 08 '17 11:08 Julio-Guerra

Hi @Julio-Guerra. Can you paste the infinit volume mount/run command you run pls?

If the cache is not activated, this can justify the bad write performance, because the root of the filesystem is updated a lot and without cache, it has to be fetched for every single write.

We also have the --async that queue edit and propagate them as fast as it can, giving way better performance. However, this is risky and can cause conflicts in certain situations. Use at your own risks.

For the record, the default block size is 1MiB.

Dimrok avatar Aug 08 '17 12:08 Dimrok

If you are using my config then change the infinit.volume to have a block_size of 1048576. This will increase your performance.

tarlano avatar Aug 09 '17 01:08 tarlano