Generate error in Chinese prompt

[linkedlist771]

The bug I was using zephyr as the models to deploy guidance as server and found it generates error when prompt contains Chinese.

To Reproduce Give a full working code snippet that can be pasted into a notebook cell or python file. Make sure to include the LLM load step so we know which model you are using.

from guidance import gen, models
import  warnings
warnings.filterwarnings("ignore")
zephra = models.Transformers("HuggingFaceH4/zephyr-7b-beta", device_map="auto")
lm = zephra + "Do you know what mushroom is? " + gen(max_tokens=20)
lm = zephra + "Do you know what 蘑菇 is? " + gen(max_tokens=20)

System info (please complete the following information):

• OS (Ubuntu.):
• Guidance Version (0.1.10):