bittensor

bittensor copied to clipboard

Description: The BTCLI is currently operating too slow for metagraph pulls and other functionality. In order to identify the core issues, we need to benchmark the various features within the CLI.

Note: It is suspected that one issue may be related to importing too many large packages.

AC:

• [ ] Setup benchmark tests to identify latency in key features of BTCLI
• [ ] Run benchmark tests
• [ ] Use results from benchmark to identify problem areas that need to be addressed with BTCLI peformance
• [ ] Draft additional tasks as problems are revealed in benchmarking

(.venv)  rooter@reef-vm ➤ ~/Downloads/testenv ➤ python -m timeit -r1 -n1 'import numpy'
1 loop, best of 1: 143 msec per loop
(.venv)  rooter@reef-vm ➤ ~/Downloads/testenv ➤ python -m timeit -r1 -n1 'import torch'
1 loop, best of 1: 1.28 sec per loop

So you can easily attribute at least 1s extra CLI delay to torch use. This is almost completely addressed by #1777 PR. I say almost completely, since I cannot guarnatee it doesn't load torch IF it is installed, as some CLI functions still use it then.