FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Unable to match the generation of demo portal with local hosting the 7b model

Open mriganktiwari opened this issue 2 years ago • 1 comments

  • I am trying to generate with zero shot prompts in a local gpu setup.
  • But with the same parameters as shown in the demo portal (i.e. temp=0.7, top_p=1.0), am not able the match the generations even in multiple trials.
  • Neither was i able to find anywhere, any guide to good generations parameters.

Any suggestions or resource would be highly appreciated.

mriganktiwari avatar May 25 '23 17:05 mriganktiwari

the same promble with local 13-b model, much bad than their demo

yuege613 avatar May 30 '23 09:05 yuege613