Parameter input rotary-freq

[jmercat]

This allows to change the rotary positional embedding frequency parameter. This is useful given the more recent approaches: LLaMA 1&2 used 10000 which is the default value here. LLaMA 3 uses 500000, Mistral uses 1000000 LLaMA long context extension works usually increase from 10000 to 100000.