BMInf
BMInf copied to clipboard
OpenBMB
Efficient Inference for Big Models
where does it optimized?
1. 您好，如图所述，我想查看 cupy操作cuda的函数的具体定义和用法，但是可能是因为cupy封装了c/c++代码，所以看不到，请问可以去哪里看呢？ 能帮忙解释一下图中第3个参数 routine 里面 4个函数执行顺序吗（我了解大概是 创建结构体、计算对称量化的scale） 跳到定义处，就只有这样的doc ------------------------ 2. 请问下图红框内为什么那样写？ ------------------------- **3. 想问一下为什么选择使用cupy直接操作cuda呢，比如allocator、igemm、fgemm的应用？这样相比使用框架（如pytorch等）实现量化有更大的好处吗？感觉cupy+cuda实现方式 要求挺高的** **非常感谢** @a710128
**Describe the bug** 使用docker环境，运行三个demo时后台都报错误 File "/usr/local/lib/python3.6/dist-packages/bminf/arch/t5/model.py", line 238, in encode True File "/usr/local/lib/python3.6/dist-packages/bminf/layers/transformer_block.py", line 42, in forward x = self.self_attention.forward(allocator, x, attention_mask, self_attn_position_bias) File "/usr/local/lib/python3.6/dist-packages/bminf/layers/attention.py", line 63, in forward qkv_i32...
**Describe the bug** 运行路径https://github.com/OpenBMB/BMInf 的demo时候，出现RuntimeError: Library cublasLt is not initialized错误 **Minimal steps to reproduce** import bminf #成功导入 cpm2 = bminf.models.CPM2() #成功定义 cpm2.fill_blank('好') #报错 RuntimeError: Library cublasLt is not initialized **Expected...
**Describe the bug** 输入: import bminf cpm2 = bminf.models.CPM2() result = cpm2.fill_blank("有一个服装品牌叫做\专门设计彩绘T恤", top_p=0.5, top_n=5, temperature=0.5, frequency_penalty=0, presence_penalty=0 ) 报错信息: Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3331, in run_code...
**Describe the bug** CUDA error was raised when importing models. This issue only happens with BMInf 1.0.x version. I could run BmInf 0.0.5 successfully. Any help would be appreciated. Thanks....
Cuda compilation tools, release 10.0, V10.0.130 torch 1.6.0 python 3.6 and get error: ------------------------------------ Traceback (most recent call last): File "/home/wac/PycharmProjects/CPM-1-Generate/test.py", line 7, in cpm2.generate(text) File "/home/wac/PycharmProjects/CPM-1-Generate/env/lib/python3.6/site-packages/bminf/models/cpm2.py", line 219, in...
**Describe the bug** how to load CPM1 model form local, now i used the following way： 1、build my model model = GPT2Model(num_layers=args.num_layers, vocab_size=args.vocab_size, hidden_size=args.hidden_size, num_attention_heads=args.num_attention_heads, embedding_dropout_prob=args.hidden_dropout, attention_dropout_prob=args.attention_dropout, output_dropout_prob=args.hidden_dropout, max_sequence_length=args.max_position_embeddings, checkpoint_activations=args.checkpoint_activations,...
