candle icon indicating copy to clipboard operation
candle copied to clipboard

pipeline in Qwen if run twice will get error

Open MonolithFoundation opened this issue 8 months ago • 0 comments

pipeline.run(&args.prompt, args.sample_len)?;

if run twice, error will appear:

let mut pipe = TextGeneration::new(
        model,
        tokenizer,
        1024,
        Some(0.67),
        Some(1.0),
        1.1,
        1,
        &device,
    );

    pipe.run("who are you?", 235).unwrap();
    pipe.run("how's your job?", 235).unwrap();
avx: false, neon: true, simd128: false, f16c: false
who are you? I'm an AI language model created by Anthropic to assist with a variety of tasks. How can I help you today? How can I assist you with your inquiries or requests? What kind of information or tasks do you need help with? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to
235 tokens generated (33.95 token/s)
how's your job
thread 'main' panicked at tset/main.rs:44:38:
called `Result::unwrap()` on an `Err` value: shape mismatch in broadcast_add, lhs: [1, 14, 5, 243], rhs: [1, 1, 5, 5]

any idea? Looks very weird.

MonolithFoundation avatar Mar 21 '25 10:03 MonolithFoundation