Prompt-Tuning icon indicating copy to clipboard operation
Prompt-Tuning copied to clipboard

Published 20 hours ago • verified publisher icon mkshing

Question of inference

Open huangjia2019 opened this issue 2 years ago • 1 comments

Hi mkshing, you mention "In the inference phase, you need to input ids to the model by using model.forward() so that you cannot use model.generate() attribute. ". Possible to elaborate more on this part? I dont really know the difference of :

  1. using model.forward(input_ids)
  2. using model(input_ids)

Thanks for help

huangjia2019 avatar Feb 07 '23 03:02 huangjia2019

Hi! I believe there’s actually no difference because model() redirects to model.forward() eventually.

mkshing avatar Feb 10 '23 11:02 mkshing