bonuschild

我下载的是release页的最新版本，信息为：```bashgost 2.11.5 (go1.19.2 linux/arm)```这个呢？应该在哪里学习呀？ ***@***.*** ---- 回复的原邮件 ---- 发件人 ***@***.***> 发送日期 2023年10月14日 16:43 收件人 ***@***.***> 抄送人 ***@***.***> , ***@***.***> 主题 Re: [ginuerzh/gost] 配置文件格式在哪里学习？ (Issue #988) v3支持命令行, yaml和json 参阅: https://latest.gost.run/getting-started/configuration-overview/ —Reply...

I've re-tested this on A100 instead of RTX3060, it show that finally it occupy about 20+GB VRAM! Why is that? I use command: ```bash python api_server.py --model path/to/7b-awq/model --port 8000...

> 1. This is normal and good - vLLM always uses nearly 100% VRAM, using the extra for caching. > 2. Sorry I've not tested AWQ with tensor parallelism on...

Thanks for reminding! I will check it soon.Message ID: ***@***.***>

@McPatate I am in Windows while #8 is in Linux. The core problem is I have no error log output to check what have happened. The `llm-ls.log` won't log anything...