Tristan
Tristan
Hi, I was trying to train a model using a RelaxedOneHotCategorical wrapped in a DistributionLambda layer on TPU. Unfortunately when using the bfloat16 TPU mixed precision option I run into...
In the styleGAN 2 paper they state using a "non-saturating logistic loss", is there any particular reason you opted for hinge loss in this implementation? For reference I believe the...
- mapping transform BEFORE windowing - var seq lens and goal tiling are computationally expensive (we ought to do on device) so I've removed from dataloader - multi-threaded tfrecords mapper...
NVAE seems to be SOTA for VAE-based models. Unlike VQ-VAE it retains the original VAE objective and is continuous. Pytorch implementation here: https://github.com/NVlabs/NVAE Main improvements are: Multi-scale architecture, separable convolution,...
Currently @sholtodouglas uses dynamic gradient clipping (gradient clipped if 4x the previous norm). This still seems to cause some issues with loss spikes. Other options are static clipping, gradient penalties,...
We hypothesis that relative actions will be easier to learn since the model does not have to learn or account for the DC component of the signal. In the literature...
Using higher dim embeddings for the rotation matrix seems to perform considerably better than quaternion or euler angle representations due to NNs learning continuous vector spaces and rotation being continuous...
Was just wondering if the JS example for interfacing with a Serum market needs updating. I've tried running it and run into the same error that the `Market.load()` expects a...
Adding conv1d layer which we hope can help with the large sequence lengths of DNA data. Has support for: - Causal convolutions (with causal padding which prevents it from looking...