llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Published 20 hours ago • verified publisher icon mosaicml

Adding temperature tuning in attention

Open ShashankMosaicML opened this issue 6 months ago • 0 comments

Adding temperature tuning in attention similar to https://github.com/huggingface/transformers/blob/9a4ce6477019358abc3ebd72d435da56f4c0ab7c/src/transformers/models/llama4/modeling_llama4.py#L332-L337

ShashankMosaicML avatar Apr 17 '25 18:04 ShashankMosaicML