Gintas Z.

Every doRpcCall times out

3

This Telegram API is confusing, already wasted 2 days. Every doRpcCall I make throws TimeoutException. If I make request doRpcCallNonAuth, it does not time out. Here is the code: apiState...

padding on moveCamera seems to be ignored

2

This was posted as a separate issue, but I didn't reply in some time and it got closed, but the problem is still there. See highlighted padding values. It seems...

Regex generation causes 37x lower performance

1

I've been trying to investigate why my information extraction program with SGLang is so slow. I've rented RTX3090 (1 x RTX 3090, 6 vCPU 26 GB RAM) and H100 (1...

/generate request possibly hanging when `CUDA out of memory` is thrown

3

I've `run_batch` with 1000 items, `num_threads=200`. I notice that the batch processing gets stuck at 98%, then the server shows no more console logs. I checked the full log and...

Llama-3 regex generation can get stuck in infinite generation beyond max_tokens and crash server (reproduction example)

2

Hey, I've just been trying to catch this bug for half a day... I've done `pip install git+https://github.com/sgl-project/sglang.git@51104cd#subdirectory=python`, which is the commit where 0.1.14 was mentioned. Launched server like this:...

I was doing `run_batch` with 50-1000 batch items, connection to a remote server, `num_threads` 10-100 and I noticed `ConnectionResetError` is sometimes thrown by this call in `http_request`: ``` resp =...

How to launch server with quantized model?

8

Sorry for a newb question, I don't find an answer. I succeeded in launching the server with unquantised Mistral7B: ``` python3 -m sglang.launch_server --model-path mistralai/Mistral-7B-Instruct-v0.2 --port 42069 --host 0.0.0.0 ```...

Feature request: persist hotkey & modifier settings

3

I enter my custom hotkey, reopen the app, and it is back to default. Also, I'd very much want to replace the default macOS "double cmd click" hotkey (which opens...

Feature request: hide app icon

1

I want to use the app running in the background but not see the app icon in the dock. An Icon in the menu bar could appear instead.