Ben Rood
Ben Rood
> 我想请问下方案4 是不是加载merge后的模型时需要向from pretrain函数传入包含nf4在内的qconfig来实现 对，就是下面这样 base_model = AutoModel.from_pretrained(config.base_model_name_or_path, quantization_config=q_config 我始终找不到 加载完 lora 后再用nf4这样的模式量化的方法，就成了我说的必须先合并再nf4量化了。
> Could this be from newlines in your shell? You might be running `./main -m /models/Meta-Llama-3-70B-Instruct.Q4_K_M.gguf -r ''` and then separately trying to run `--in-prefix "\nuser\n\n"` and so on. What...
I also want this feature be implemented. I've made a dirty hack, which solve errors on server side, but I'm not sure the stats_correct is OK after this hack. ```...
I also got this error in current version.
> @bash99 we have a long standing PR #2014 for this issue. Let me see if we can rebase and reprioritize it. Yes, this patch fix it for me. Does...
Some amends for @madeye reply how do they attack the users? Identify and block IPs. Beside this attack, they can block the special port of that ip for a while...
80 port with obfs-http seems by detected by local ISP (tls is dead long time ago). After running simple-obfs and ss-server (AHED) for a while, 80 port is blocked, close...
> > 80 port with obfs-http seems by detected by local ISP (tls is dead long time ago). > > After running simple-obfs and ss-server (AHED) for a while, 80...
use non-standard port like 80 or 443, recent GFW are more strict, I think it will block 80/443 traffic for some IP blocks (I've a vps is blocked on port...
And open chat ui failed in local install with same error.