FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 3765 closing signal SIGTERM WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 3766 closing signal SIGTERM ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -7) local_rank: 2 (pid: 3767) of binary: /usr/bin/python3 Traceback (most recent call last): File "/usr/local/bin/torchrun", line...
I've been working on a ProgrammaticChatIO class to enable control of FastChat by other programs through standard streams (stdin/stdout/stderr). This was motivated by a desire to use it in JavaScript...
I want to fine-tuning vicuna via train_mem.py. It requires module flash_attn, and flash_attn requires nvcc. Based on that, I assume train_mem.py can only run on GPU. Is my understanding right?...
While tuning I am getting the following error. AssertionError: No inf checks were recorded for this optimizer. Can anyone help me with this? Here are my training arguments: per_device_train_batch_size=2, warmup_steps=100,...
controller:  model_worker: I waited for an hour with no output and no errors. Can someone tell me what's going on?
Now prompt is cut to `max_src_len`, so prompt + new tokens is less than context_len. Then a lot prompt would be excluded, like system prompt. Could it use a sliding...
execute command: python -m fastchat.serve.cli --model-path ./model_weights/lmsys/vicuna-7b-delta-v1.1 --load-8bit error content: Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 2/2 [00:04 Tensor: │ │ │ │ D:\python\Lib\site-packages\torch\cuda\__init__.py:239 in _lazy_init │ │ │ │ 236 │...
when running apply_delta.py of weight version v1.1, I find that the the shape of params lm_head.wegiht is [32000, 5120] in base weight, and is [32001, 5120] in delta weight. However,...
Kubuntu 22.10, Nvidia driver Version: 530.30.02, CUDA Version: 12.1, RTX 3080 12G When the gradio webpage is left open for a few seconds after the response, the bot does not...
