sixgod
sixgod
该问题已经在最新的模型文件中修复，请在huggingface下载最新的模型文件或参考修改https://huggingface.co/THUDM/LongWriter-glm4-9b/commit/778b5712634889f5123d6c463ca383bc6dd5c621
在模型文件中，`tokenization_chatglm.py`文件，`ChatGLM4Tokenizer`类的`_pad`函数的传入参数中加一条`padding_side`: Optional[str] = None, ```Python def _pad( self, encoded_inputs: Union[Dict[str, EncodedInput], BatchEncoding], max_length: Optional[int] = None, padding_side: str = "left", padding_strategy: PaddingStrategy = PaddingStrategy.DO_NOT_PAD, pad_to_multiple_of: Optional[int] = None, return_attention_mask: Optional[bool]...
Please refer to this issue：https://github.com/THUDM/GLM-4/issues/578
你这个输出是不是不完整
vllm要用最新版
启动参数就这些[vllm_openai_server.py](https://github.com/THUDM/CogAgent/blob/main/app/vllm_openai_server.py)： ```Python def load_model(model_dir: str): engine_args = AsyncEngineArgs( model=model_dir, tensor_parallel_size=1, dtype="bfloat16", trust_remote_code=True, gpu_memory_utilization=0.9, enforce_eager=True, worker_use_ray=True, disable_log_requests=True, # 如果遇见 OOM 现象，建议开启下述参数 # enable_chunked_prefill=True, # max_num_batched_tokens=8192 ) engine = AsyncLLMEngine.from_engine_args(engine_args) return engine...
> is CogAgent support vLLM officially？ sure
> 使用vllm的版本到底是多少呀，我用0.7.2以及0.6.6都不行 理论上0.6.6以后都可以，你遇到什么问题，可以新开一个issue讨论
目前仅支持action_space中的操作
This repository is not an open source version of autoglm. For the latest autoglm product information, please visit the official website: https://agent.aminer.cn/