Windkit Li

## Further Analysis ### Object Timestamp (Statistics gathered with `leofs-adm whereis` of entire key domain) | Time | Number of Entries | | --- | --- | | 11:25:16 |...

## Further Analysis (Test with Processing Time Logging) ### Preparation Processing Time is logged at `leo_gateway` and `leo_storage` to check the request flow 1. `leo_gateway` - Duration spent on put...

## Investigation To find out the root cause of the problem ## Methodology The process of storing an object in `leo_object_storage` involves two types of Disk IOs 1. Metadata Related...

## Investigation Check if the backend db is a bottleneck ### Methodology Relocate the backend DB to memory ``` bash $ mkdir /dev/shm/metadata $ ln -s /dev/shm/metadata /ssd/avs/metadata ``` ###...

After loading the 4mil image dataset, each storage node holds 2.5mil objects and metadata disk usage is around 200MB ``` $ du -sh /dev/shm/metadata 200M /dev/shm/metadata ```

@mocchira Thanks for the comment, I will test it today

## LeoFS's LevelDB Settings seem to be too conservative ### Description To begin with, I checked `eleveldb` and `leveldb` projects for clues. At [basho/leveldb](https://github.com/basho/leveldb), the team suggests settings as they...

**Max Open File: 1000, Write Buffer Size: 60MB  **Max Open File: 10000, Write Buffer Size: 600MB  The results are similar despite of the 10x difference in parameters, there...

 Setting the `{fadvise_willneed, true}` actually helps to overcome the IO spikes by using the fadvise hint. Yet the throttling still occurs from time to time, it is observed that...

PR for fadvise_willneed in eleveldb https://github.com/leo-project/leo_backend_db/pull/16