PoHaoYen

Problems encountered during speculative decoding execution

1

Hi, I attempted to use speculative decoding but encountered some errors. May I ask for your assistance? I used the parameters from the first example. python ./examples/speculative_inference.py \ --model gpt2-xl...