jsonformer icon indicating copy to clipboard operation
jsonformer copied to clipboard

allow for do_sample=True

Open chris-aeviator opened this issue 2 years ago • 1 comments

we might not always want greedy sampling do we? Could you implement do_sample beeing an init param for JsonFormer or is there anything technically that prohibits this change?

chris-aeviator avatar Aug 22 '23 13:08 chris-aeviator

I was able to set my model to do_sample=True before passing it to jsonformer. model = AutoModelForCausalLM.from_pretrained( "databricks/dolly-v2-3b", device_map='auto', use_cache=True, do_sample=True)

lettucegoblin avatar Oct 04 '23 20:10 lettucegoblin