SwissArmyTransformer
SwissArmyTransformer copied to clipboard
THUDM
SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.
尝试安装时候出现以下错误 ``` PS D:\deeplearning\VisualGLM-6B> pip install SwissArmyTransformer>=0.4.4 Collecting SwissArmyTransformer>=0.4.4 Obtaining dependency information for SwissArmyTransformer>=0.4.4 from https://files.pythonhosted.org/packages/91/b9/44a3e9cc0116a3ebf8b60f0ef67340ca22c195e37b9d8f951a37e114f300/SwissArmyTransformer-0.4.4-py3-none-any.whl.metadata Using cached SwissArmyTransformer-0.4.4-py3-none-any.whl.metadata (10 kB) Requirement already satisfied: torch in c:\python311\lib\site-packages (from SwissArmyTransformer>=0.4.4) (2.0.1+cu118)...
logits, *mems = model(inputs_ids, position_ids, attention_mask) # print(logits.shape) loss_func = CrossEntropyLoss(ignore_index=-100) loss = loss_func(logits.view(-1, logits.size(-1)).float(), labels.view(-1))`` 我是这样写的loss计算方式，会出现一个/opt/conda/conda-bld/pytorch_1670525539683/work/aten/src/ATen/native/cuda/Loss.cu:242: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [15,0,0] Assertion `t >= 0 && t < n_classes`...
加载方式 ``` model = AutoModel.from_pretrained(visualchatglm_model_path,trust_remote_code=True).to(torch.cuda.current_device()) ``` 运行环境如下： ``` SwissArmyTransformer 0.3.7 transformers 4.28.1 deepspeed 0.9.1 torch 1.11.0+cu113 torchaudio 0.11.0+rocm4.5.2 torchvision 0.12.0+cu113 cpm-kernels 1.0.11 einops 0.6.1 ``` 报错日志如下： ``` Epoch_0: 0%| |...
I did not find such a cached method using past_key_values in the SAT. Is it possible to add this? Thanks.
直接跑源码的qlora.py，报错  给model.child = LoraLinear(100, 200, 10)改为model.child = LoraLinear(100, 200, 10，10，2)后，又报错 
## What I did I pass the "-h" option to print the help message. However, my script complains `TypeError: %o format: an integer is required, not dict` in `/usr/lib/python3.8/argparse.py:Line633`. The...
[2023-07-06 16:43:31,720] [INFO] [RANK 0] Try to load tokenizer from Huggingface transformers... Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code...
在镜像中安装报错
Collecting SwissArmyTransformer Using cached SwissArmyTransformer-0.3.7-py3-none-any.whl (2.4 MB) Requirement already satisfied: torch in /usr/local/lib/python3.8/site-packages (from SwissArmyTransformer) (1.13.1) Requirement already satisfied: transformers in /usr/local/lib/python3.8/site-packages (from SwissArmyTransformer) (4.27.1) Requirement already satisfied: sentencepiece in...
