Tarun Agarwal

[Feature Request] Quadratic Mean

1

Quadratic mean or RMS is a very popular measure which seems to be missing. [Wikipedia page for Quadratic mean](https://en.wikipedia.org/wiki/Root_mean_square)

Closes #233 **Changes** - Add Seq.meanQuadratic - Add Seq.meanQuadraticBy **Description** This PR adds the Quadratic mean (also known as RMS). Here is the [wikipedia page for the Quadratic mean](https://en.wikipedia.org/wiki/Root_mean_square). -...

KeyError: 'model.embed_tokens.weight' when using mergekit-moe

5

I am getting the following error when I run `mergekit-moe config.yml ./output` ``` File "/home/ubuntu/moe_experiments/mergekit/mergekit/options.py", line 58, in wrapper f(*args, **kwargs) File "/home/ubuntu//moe_experiments/mergekit/mergekit/scripts/mixtral_moe.py", line 394, in main build( File "/home/ubuntu//moe_experiments/mergekit/mergekit/scripts/mixtral_moe.py",...

Fixing a typo in the claude 3 blog post.