Paweł Gadziński

# Description Meta released LLama 3 model in April. We have tutorial for Llama 2. It turned out that it works with Llama 3. I changed comments within tutorial. They...

Generation tutorial for Gemma model

8

# Description I added the tutorials with finetuning and with generation for the Gemma model. Moreover I added few features that were neccessary to make my tutorials work. ## Type...

[PyTorch] Fixed bug with loading calibrated weights

11

I encountered the bug related to calibration of scaling factors and loading the model. When I calibrate the scaling factors with model with weights in bf16, the model parameters are...

Different dimension for attention

3

# Description I added `channels_per_head` parameter to attention. I know that it is breaking change, but I think there is not a good way of doing it - look at...

Tests for distributed

1

# Description I am working on debug API. Before it can be merged, it needs to be tested. We need to ensure that all the new layers will work properly...