mamba icon indicating copy to clipboard operation
mamba copied to clipboard

Add inputs_embeds as alternative for input_ids

Open Maykeye opened this issue 1 year ago • 0 comments

Adding to the model.forward inputs_embeds (same name as HF models use) to give the model token embedding directly rather than token ids

The main use case is training soft prompts without adding N tokens to tokenizer that never get outputted(and then jumping through hoops to train them only without touching existing tokens)

(ETA: from more recent example: techniques like neft are much easier to do when it's possible to pass embedding directly)

Maykeye avatar Feb 03 '24 12:02 Maykeye