Nikola Borisov

More metrics

2

Adding some more metrics to Prometheus. Total Number of requests Total Number of input tokens Total Number of output tokens Avg time per output token Max time per output token...

Use single connection instead of connection pool

1

Hello, I was kind of wondering why redis-py is using a connection pool instead of using a single connection for all the requests to redis. Since redis is single threaded...

htop too slow on large server

12

When I try using htop on a server with around 200 cores and 2TB of memory (H100x8 server) htop is super super slow. It shows black screen for about 60s...