HeroSong666

Better inference based on starcode2-3b model

1

I am new to starcode. when I run the follow demo: ``` import torch from transformers import AutoTokenizer, AutoModelForCausalLM checkpoint = "./starcoder2-3b" tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16)...

关于语音合成TTS的中英混合模型

4

我在https://github.com/k2-fsa/sherpa-onnx/releases/tag/tts-models里面好像没有找到中英混合模型，请问后续会添加中英混合的模型吗？thx

关于文本摘要生成算法中，输入字数限制的问题

3

我在input输入大段的新闻内容时会被截断，但是找不到字数限制的参数，作者可以指个路吗？谢谢！

运行下面单元格时： torch.set_default_tensor_type(torch.cuda.HalfTensor) tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) model = AutoModel.from_pretrained( "THUDM/chatglm-6b", trust_remote_code=True, device_map=DeviceMap("ChatGLM").get() ) 出现报错： AttributeError Traceback (most recent call last) [](https://localhost:8080/#) in () 1 torch.set_default_tensor_type(torch.cuda.HalfTensor) 2 ----> 3 tokenizer =...

An API that can connect the code model and the chat model at the same time (just like Playground)

3

The API currently provided can only connect to the code model（https://tabby.tabbyml.com/api）. Is there an API that can access the code model and chat model at the same time like Playground?

Error occurs about tabby-python-client

3

when i follow the steps in: https://github.com/TabbyML/tabby/tree/a07a070fa97c270b192151e14fd60b538ef666ff/python/tabby-eval/modal/tabby_python_client , i run the following code in the path: /tabby-main/python/tabby-eval/modal/tabby_python_client/: from tabby_python_client import AuthenticatedClient client = AuthenticatedClient(base_url="https://api.example.com/", token="SuperSecretToken") （replaced by my url and...

您好，在运行过程中出现了如下的问题，请问是不是gensim的版本问题？该如何解决呢？

1

TypeError Traceback (most recent call last) Cell In[18], line 13 9 return "".join(result_sentence) 11 text = codecs.open('C:/Users/hs6/Desktop/03.txt', 'r', 'utf-8').read() ---> 13 result_data = do(text, 3) 14 print(result_data) Cell In[18], line...

Better inference based on starcode2-3b model

1

I am new to starcode. when I run the follow demo: ``` import torch from transformers import AutoTokenizer, AutoModelForCausalLM checkpoint = "./starcoder2-3b" tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16)...

Make full use of all GPU resources for inference

7

### What is the issue? I used 4 A30 gpus to reason about qwen2-72b's model. But even at peak times, each card was not used more than 35%. At the...

ollama does not work on ALL GPU automatically

4

### What is the issue? when I use the ollama:0.1.38, I use the following command to start: ` docker run -d --gpus=all -v /root/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama `...