MOSS

MOSS copied to clipboard

请问moss-003-pm-data偏好数据什么时间能开源？

2

如题

liumingzhu6060

请问参数do_sample=True，推理结果中还是有不断重复输出是为什么

zhaoyang-coder

扩展文本长度失败

4

在config配置中更新相关参数 ``` config = AutoConfig.from_pretrained(model_path, trust_remote_code=True) config.update({"n_positions": 4000}) config.update({"max_position_embeddings": 4000}) model = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, load_in_8bit=True, device_map='auto', torch_dtype=torch.float16, config=config ) ``` 可以发现[self.causal_mask](https://huggingface.co/fnlp/moss-moon-003-sft/blob/7119d446173035561f40977fb9cb999995bb7517/modeling_moss.py#L63)成功初始化为[1, 1, 4000, 4000]. 然而在[_att](https://huggingface.co/fnlp/moss-moon-003-sft/blob/7119d446173035561f40977fb9cb999995bb7517/modeling_moss.py#L107)中再次打印self.causal_mask，shape会重新变为[1, 1, 2048, 2048] 导致尺寸不匹配问题。...

JaheimLee

RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1)

1

使用moss-base-7b,两张3090部署时,报错: > RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1)   相关代码 ``` import os import torch from huggingface_hub import...

lizhixi212

how to run fnlp/moss-base-7b in multi gpus, i use this way: from transformers import AutoTokenizer, AutoModelForCausalLM **os.environ['CUDA_VISIBLE_DEVICES'] = "0,1"** tokenizer = AutoTokenizer.from_pretrained("fnlp/moss-base-7b", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("fnlp/moss-base-7b", trust_remote_code=True).cuda() model = model.eval()...

FakerYFX

RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might...

whay181

Thanks for your wonderful work. The bottle-neck of MOSS may lie in datasets used in the pretrain phase. We want to continue pretrain MOSS on multi datasets like 悟道, Wikipedia...

h-ccc

两张v100部署失败

2

本人环境 两张v100（32G*2） cuda11.0 pytorch版本 1.7.1 由于pytorch版本比较低，无法支持量化版本，所以选择部署fnlp/moss-moon-003-sft这个模型，但是fp16精度会报以下错 `File "/root/anaconda3/envs/mossgpu/lib/python3.8/site-packages/torch/tensor.py", line 547, in __rpow__ return torch.tensor(other, dtype=dtype, device=self.device) ** self RuntimeError: "pow" not implemented for 'Half'` 所以只好改成 ` raw_model = MossForCausalLM._from_config(config, torch_dtype=torch.float32)...

Cocoalate

请问如何基于Langchian+moss模型构建本地知识库问答

secslim

cuda error

1

 ``` ../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [0,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed. ``` 这个错误是什么情况 torch 2.0.0 cuda11.8 moss-moon-003-sft-int8

wqh17101