truss icon indicating copy to clipboard operation
truss copied to clipboard
verified publisher icon basetenlabs

b10_lookahead speculation mode

Open mahmoudhas opened this issue 5 months ago • 0 comments

Adds speculation mode BASETEN_LOOKAHEAD as an alias to:

    speculator:
      speculative_decoding_mode: LOOKAHEAD_DECODING
      enable_b10_lookahead: true
      lookahead_windows_size: 1
      lookahead_ngram_size: 32
      lookahead_verification_set_size: 1

mahmoudhas avatar Jul 30 '25 16:07 mahmoudhas