UniEon

Results 3 comments of UniEon

Error in lxc image copy after upgrade to 4.19

Result for `lxc remote list`: ``` +-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+ | NAME | URL | PROTOCOL | AUTH TYPE | PUBLIC | STATIC | GLOBAL | +-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+ | images | https://images.linuxcontainers.org | simplestreams...

OpenAI timeout

Facing the same issue on self hosted LLM running gemma-2-9B-Instruct with fastchat and sglang worker, which works when token is set to 512 but timeout when max_token is 2000. When...

Single GPU vs multiple GPU (tensor parallel) suggestion for API Server

If you use 4x4060ti the speed of inference might be slow given the communication overhead between the GPU's (not tried a benchmark but just based on theoretical knowledge). If you...