Clarification on Memory Optimization with -b and -c Parameters

[SaimMomin12]

Hi,

Thank you for a much awaited version of DIAMOND with all the potential bug fixes. Appreciate it.

I am currently working on optimizing the memory requirements for DIAMOND on our Galaxy server to ensure it runs efficiently for our users. While going through the documentation, I came across the -b (--block-size) and -c (--index-chunks) parameters, which seem to play a key role in controlling memory usage. However, I would appreciate some clarification on how these parameters influence memory usage and how to calculate the memory requirements when using them.

Specifically, I have the following question:

1. How do -b and -c influence memory usage? (Could you explain how these parameters interact with the database and query sequences to determine the overall memory footprint? Could you provide an example calculation?)

Your insights would be incredibly helpful in ensuring that we configure DIAMOND optimally for our Galaxy server, balancing memory usage and performance for our users.

Thank you!

Best, Saim

[bbuchfink]

The memory use can be expected to be roughly 18b/c + 2b (in gigabytes). With enough memory it is first recommended to use c=1 and then increase the block size. Increases in block size improve the performance (less for modes with higher sensitivity), but that also saturates at some point and exceedingly high block sizes are not recommended.