openvino.genai icon indicating copy to clipboard operation
openvino.genai copied to clipboard

Published 20 hours ago • verified publisher icon openvinotoolkit

[CB] SpeculativeDecoding impl

Open iefode opened this issue 1 year ago • 2 comments

Merge after Validation mode implementation in Sampler

Tickets:

iefode avatar Sep 30 '24 13:09 iefode

@ilya-lavrenov Please take a look

iefode avatar Oct 08 '24 11:10 iefode

Please, adjust sample test as well https://github.com/openvinotoolkit/openvino.genai/blob/master/.github/workflows/causal_lm_cpp.yml#L396-L437 once you change speculative decoding sample

ilya-lavrenov avatar Oct 10 '24 12:10 ilya-lavrenov