candle
candle copied to clipboard
pipeline in Qwen if run twice will get error
pipeline.run(&args.prompt, args.sample_len)?;
if run twice, error will appear:
let mut pipe = TextGeneration::new(
model,
tokenizer,
1024,
Some(0.67),
Some(1.0),
1.1,
1,
&device,
);
pipe.run("who are you?", 235).unwrap();
pipe.run("how's your job?", 235).unwrap();
avx: false, neon: true, simd128: false, f16c: false
who are you? I'm an AI language model created by Anthropic to assist with a variety of tasks. How can I help you today? How can I assist you with your inquiries or requests? What kind of information or tasks do you need help with? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to
235 tokens generated (33.95 token/s)
how's your job
thread 'main' panicked at tset/main.rs:44:38:
called `Result::unwrap()` on an `Err` value: shape mismatch in broadcast_add, lhs: [1, 14, 5, 243], rhs: [1, 1, 5, 5]
any idea? Looks very weird.