TianTengya

llm_perf: generate config中 eos_token_id 配置问题

1

https://github.com/bytedance/ByteMLPerf/blob/main/byte_infer_perf/llm_perf/server/endpoint.py ```python # create generate config config = GenerateConfig( min_new_tokens=generate_config.get("min_new_tokens", 0), max_new_tokens=generate_config.get("max_new_tokens", 0), top_k=generate_config.get("top_k", 0), top_p=generate_config.get("top_p", 1.0), temperature=generate_config.get("temperature", 1.0), presence_penalty=generate_config.get("presence_penalty", 1.0), _eos_token_id=self.tokenizer.eos_token_id,_ pad_token_id=self.tokenizer.pad_token_id, get_input_logits=bool(generate_config.get("get_input_logits", 0)), ) req = GenerateRequest(input_ids, config)...

Exception: module 'triton' has no attribute 'set_allocator'

1

my env as follow: torch==2.6.0 fbgemm_gpu==1.1.0 torchrec==1.1.0 gin_config==0.5.0 pandas==2.2.0 tensorboard==2.19.0 pybind11