Clay Mullis

@janEbert end of an era, eh?

@lucidrains @rom1504 Has some stability issues surrounding the top_k function I think. Without DeepSpeed to auto-skip NaN's in Pytorch native, training can break after awhile. This was alleviated quite a...

@lucidrains I'm working with some modifications to my code; line numbers are inaccurate. Looking at the loss graph now it's pretty obvious there's a trend toward increasing in loss before...

> ok I confirm this code is working with torch 1.10, however one drawback is it increases the vram usage (because it's loaded vqgan as float32 instead of float16) I...

Sorry this only effects non-distributed pytorch. Are you using 16-bit precision with DeepSpeed? My current impl of mixed precision for pytorch was enabled by default. Due to stability issues I've...

> do you want to see if 1.1.5 fixes this? https://github.com/lucidrains/DALLE-pytorch/releases/tag/1.1.5 @lucidrains Yes that stabilized the training thanks

Hm - it looks like the dtype specifier isn't available on pytorch LTS. Must be new. I don't know of another way to solve the issue (for deepspeed), however. It...

Yes, I would be somewhat more comfortable with a hard requirement on Pytorch 1.10 if it didn't also mean a harsh decision of only CUDA 11.3 (unavailable for my operating...

> Yes several people did that, but nobody packaged a vqgan inference script properly, it would be useful to do I'm considering a package similar to your clip-retrieval repo centered...

> Looks like issues are accumulating. Some of them are already answered. > I think it's nice when there are fewer open issues. > Let's close them ? > >...