Carlos Mocholí
Carlos Mocholí
Thanks for the link @ananthsub. For anybody reading, this is how you would use it: ```python import torch from torcheval.tools.module_summary import get_module_summary from pytorch_lightning.demos.boring_classes import BoringModel model = BoringModel() summary...
A FLOP counter was added to PyTorch: https://github.com/pytorch/pytorch/pull/95751
#18848 added this small utility (to be released with 2.2) ```python from lightning.fabric.utilities import measure_flops with torch.device("meta"): model = MyModel() x = torch.randn(2, 32) model_fwd = lambda: model(x) fwd_flops =...
@alstonlo My bad! Let me fix that quickly
Opened https://github.com/Lightning-AI/lightning/pull/18085
One way to reduce the surface of issues would be to do ```python import lightning as L from lightning.pytorch.loggers.wandb import WandbLogger def fn(fabric, logger): ... logger = WandbLogger() fabric =...
I'll take a look. Thanks for the report!
Transfering this to lightning since this file no longer exists, but there's still an underlying bug
How are you configuring fp16? I had the same thing happen to me because I was using `torch.set_default_dtype(torch.float16)` to enable fp16 and that was being ignored with the TE layers...
Maybe ignore my above comment, I'm actually seeing the same problem. It seems to be exacerbated by the number of TE layers used. On an LLM, when I replace [only...