jsonformer
jsonformer copied to clipboard
allow for do_sample=True
we might not always want greedy sampling do we? Could you implement do_sample beeing an init param for JsonFormer or is there anything technically that prohibits this change?
I was able to set my model to do_sample=True before passing it to jsonformer.
model = AutoModelForCausalLM.from_pretrained( "databricks/dolly-v2-3b", device_map='auto', use_cache=True, do_sample=True)