cornzz

RoPE is only applied to queries and keys, this is how it was introduced in the RoFormer paper (section 3.2): https://arxiv.org/pdf/2104.09864 In fact, even before RoPE, relative position embedding was...

@patrickvonplaten I also noticed this, I agree it should be OR not XOR. Created a PR https://github.com/mistralai/mistral-inference/pull/214

I believe notion changed the domain of their user content servers so images are not located at`www.notion.so/img` anymore, but `img.notionusercontent.com`, which the original url redirects to now. The right fix...

Fix here --> https://github.com/asnunes/notion-page-to-html/pull/42

Hi, any update? 😅

Edit: **TLDR** - the model (downloaded from mistralcdn and using the mistral-inference library) starts generating nonsense (see below) at sequence lenghts of 2000-2300 tokens, which is far below the theoretical...

@iofu728, @XufangLuo I had to [adjust](https://github.com/microsoft/LLMLingua/pull/182/commits/518204df76596111f0eff9ae9793ccdeef74f6c0) the llmlingua2 tests slightly as the generated compressed text now differs a bit from before.

@patrickvonplaten Same problem for `mistralai/Mistral-7B-v0.1`, the tokenizer file is called `tokenizer.model` but the regex is looking for a file ending with `\.v.*` https://huggingface.co/mistralai/Mistral-7B-v0.1/tree/main

Quick fix for me was going into ``` ~/.cache/hub/huggingface/hub/models--mistralai--Mistral-7B-v0.1/snapshots/7231864981174d9bee8c7687c24c8344414eae6b ``` and creating a new link to the tokenizer file ``` cp -P tokenizer.model tokenizer.model.v1 ``` then setting `export HF_HUB_OFFLINE=1` and...

This is exactly what we also need, thanks a lot @LukasWillin! @mojoaxel / @yotamberk could you merge this so we dont have to create our own fork of vis-timeline with...