How to use the python binding?

[sunxichen]

I successfully installed it from source, but I don't know how to use the chat and stream_chat interfaces, or how to load the model, etc. Are there any documents that I can refer to?

[xwdreamer]

how do you convert the model。

[sunxichen]

how do you convert the model。

Just follow the instruction in README. I converted the model in Intel Mac, not M1, and there is no error. I'm using transformers==4.33.2, maybe check your transformers package version?

[simonJJJ]

Now only supports tiktoken_cpp python bindings. I will release more python bindings within this week. Stay tuned.

[quan2005]

Mark, I need Python bindings too.

[simonJJJ]

Python bindings are available now.

[AppleJunJiang]

Python bindings are available now.

怎么使用，没有在文档里面看到，目前只有tiktoken_cpp python bindings

[yuebo]

You can use below codes in python

from qwen_cpp import Pipeline                 
pipeline = Pipeline("/path_to_models/qwen7b-ggml.bin", "/path_to_tiktoken/Qwen-7B-Chat/qwen.tiktoken")

result1 = pipeline.chat(["Hello"])
print(result1)
result2 = pipeline.chat(["Hello"],stream=True)
for item in result2:
  print(item)

[cailuyu]

Seem python-bind did not support ”system”！ Anyway to support it? it should be easy to change the tokenizer to support

[zzzcccxx]

You can use below codes in python

from qwen_cpp import Pipeline                 
pipeline = Pipeline("/path_to_models/qwen7b-ggml.bin", "/path_to_tiktoken/Qwen-7B-Chat/qwen.tiktoken")

result1 = pipeline.chat(["Hello"])
print(result1)
result2 = pipeline.chat(["Hello"],stream=True)
for item in result2:
  print(item)

@yuebo 请问可以指定只在cpu上运行吗？