Carlos Mocholí

Also, I suggest that you check out https://github.com/Lightning-AI/lit-gpt for all your gpt training needs. The one you linked is an earlier version that hasn't been updated

@joseph35533553 We have adapter merged (https://github.com/Lightning-AI/lit-parrot/blob/main/howto/finetune_adapter.md), and Lora on its way: #128

Thanks for the report. Unfortunately, PyTorch doesn't support this, so we cannot measure the flops used by Mixtral the way we do. For the moment, you can avoid the error...

Hey! All your suggestions make sense to me. You should be able to split the combined ff linear as you suggest, especially if load_param has ben called already. We also...

I would strongly prefer that we don't add this new MLP class. To debug the output, you'll have to inspect the activations for both models layer by layer to see...

Hi! I don't think this should happen. Can you share the exact command that you ran, the complete error stacktrace, and any changes you made to the repository?

Sorry, it was an accident!

Quantization support landed with #104 so now you can do inference with less requirements.

For the moment, and as a workaround, I would strongly suggest that you simply `torch.compile` the underlying `nn.Module` instead of the `LightningModule`

Here's a repro ```python import os import torch from lightning.pytorch import LightningModule, Trainer from torch.utils.data import DataLoader, Dataset from lightning.pytorch.demos.boring_classes import RandomDataset class BoringModel(LightningModule): def __init__(self): super().__init__() self.layer = torch.nn.Linear(32,...