zhouyonglong

改成和2.1.3版本一致就可以了，少传入attn_mask这个参数 def infer(self, inputs, attn_mask): **last_hidden_states = self.ls_bert.infer(inputs, _attn_mask_)** last_hidden_states = torch.Tensor(last_hidden_states).float() pooled_output = self.pooler(last_hidden_states.to("cuda:0")) logits = self.classifier(pooled_output) return logits 但是lightseq性能还不huggingface未加速版本，GPU是 1080Ti ====================END warmup==================== tokenizing the sentences... =========lightseq========= lightseq generating......

我也用的是1080ti，cuda11.0，tensorflow2.4，在example里面的bart测试上也不如huggingface版本

https://github.com/zhouyonglong/Fast-Python-Collaborative-Filtering-for-Implicit-Datasets/blob/master/implicit/als.py里面有一个explain函数

Traceback (most recent call last): File "/home/llm/mathGlm/MathGLM-main/MathGLM_MWP/inference_mathglm.py", line 124, in main(args) File "/home/llm/mathGlm/MathGLM-main/MathGLM_MWP/inference_mathglm.py", line 38, in main model, args = CachedAutoregressiveModel.from_pretrained(model_path,args) File "/mnt/disk1/anaconda3/envs/mathglm/lib/python3.9/site-packages/sat/model/base_model.py", line 216, in from_pretrained return cls.from_pretrained_base(name, args=args,...

load模型的时候加一个参数，build_only=True，可以解决你提到的这个问题 model, args = CachedAutoregressiveModel.from_pretrained(model_path,args,build_only=True) 但是又会出现新的问题 ^@[2023-10-03 09:08:14,117] [INFO] [RANK 0] Cannot find THUDM/chatglm2-6b from Huggingface or sat. Creating a fake tokenizer... Traceback (most recent call last): File "/home/llm/mathGlm/MathGLM-main/MathGLM_MWP/inference_mathglm.py", line...