Clint-chan

wtf.

LOG_DIR="/raid/xinference/modelscope/hub/qwen/Qwen2-72B-Instruct/logs"; \ mkdir -p "$LOG_DIR"; \ LOG_FILE="$LOG_DIR/vllm_$(date +%Y%m%d).log"; \ echo "Logging to $LOG_FILE" >&2; \ conda run -n vllm_062 vllm serve \ /raid/Qwen2-72B-Instruct \ --port 20005 \ --dtype bfloat16 \...

分析一下我的日志：INFO 09-29 08:48:42 logger.py:36] Received request chat-312348aecf9e42ec91de81da41e091db: prompt: 'system\n你是一位商品名词分析专家，请从描述内容中分析是否包含与输入的产品关键词含义相同。请使用Json 的格式返回如下结果，结果的值只允许有包含和不包含。格式 如下：{“result”:"结果"}\nuser\nFIBER CABLE是否包含在哪下内容中，内容如下：None\nassistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.7, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False,...

i face a new bug now, INFO: 61.171.72.231:10464 - "POST /v1/chat/completions HTTP/1.1" 200 OK INFO 10-16 11:05:06 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241016-110506.pkl... [rank0]:[E1016 11:05:06.676414458 ProcessGroupNCCL.cpp:1515] [PG 3...

 still