optimum icon indicating copy to clipboard operation
optimum copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

what the difference between decoder_model.onnx and decoder_with_past_model.onnx

Open akk-123 opened this issue 3 years ago • 1 comments

Feature request

none

Motivation

none

Your contribution

none

akk-123 avatar Sep 22 '22 03:09 akk-123

Hi @akk-123,

The difference is that the decoder_with_past_model.onnx has the pre-computed key/values hidden-states as one of its inputs while the decoder_model.onnx has not. See here for more information.

echarlaix avatar Sep 22 '22 08:09 echarlaix

@echarlaix The link is not available anymore, can you please update your last comment?

ayansengupta17 avatar Jan 24 '23 16:01 ayansengupta17

@ayansengupta17 It has been moved here.

regisss avatar Jan 24 '23 17:01 regisss