Mohit Lamba

How to choose wd?

2

Thankyou for this wonderful benchmarking. In several experiments `wd=1.2e-6`. Can you please give some guidelines or rule of thumb in choosing the hyperparameter for weight decay?

Modifying Eq 12/13 to deal with negative loss

1

Hi, Thanks for this great work. In Eq. 12/13 you have a `log(variance)` that might go negative for variance

Thankyou for this project but I was interested to know how can we use the siren activation with any other deep model such as ResNet, DenseNet, etc? Currently, I guess...

Change the width of the post's/page's content

1

Is there an easy way to change the width of the post's/page's content? The default width of the content is too narrow for me. I expect it to be somewhere...

Compatibility to Python 3

1

Thank you for this nice code. I observe that it has been written for Python2. Is there a version compatible for Python3? Thankyou

`torch.nn.utils.weight_norm` does the crucial weight re-parametrization during training, but does it happen during testing also? If yes should not they be removed before proceeding for testing?

Hello, I noticed you comment mentioning that Clipping Flow values helped you a lot and default value you set to (-3,3), https://github.com/lucidrains/rectified-flow-pytorch/blob/1f59e7e9122439931968126d4ac662c1f0ce24ac/rectified_flow_pytorch/rectified_flow.py#L159 My question is about the range of Clipping...

Hello, Thankyou for detailed and yet easy to follow implementation of Rectified Flow paper. I was going through the original implementation by authors and the authors use the original LPIPS...