Julien Hauret

icon indicating a website link https://jhauret.wordpress.com/

icon company Conservatoire National des Arts et Métiers icon location Paris icon location PhD Student - Deep Learning & Speech Processing

Results 12 comments of Julien Hauret

LightningModule self.log add_dataloader_idx doesn't reduce properly the metric across dataloaders

Same issue, when using `self.log("my_metric", value, add_dataloader_idx=False)` in the `validation_step` of the `LightningModule` and the `val_dataloader` method of the `LightningDataModule` look like this: ```python def val_dataloader(self): dataloader_A = DataLoader(...) dataloader_B...

The Positional Encoding is not using sin / cos?

From [GPT-1 paper](https://www.mikecaptain.com/resources/pdf/GPT-1.pdf): > We used learned position embeddings instead of the sinusoidal version proposed in the original work.