Clay Mullis

I've done a run using `--loss_img_weight 1` and setting the presently hidden `stable` parameter to `True` in the DALLE initialization. Here is a W&B report. I'm not tracking text and...

Here is the byte pair encoding I used. Vocab size of 8192 covering 99.999% of all unique characters in about 6 million captions from conceptual captions. Perhaps overkill for these...

Fun experiment - didn't really pan out. There seems to be a mild tradeoff where the generations match maybe just one word in the caption rather than the full caption....

@janEbert is the relation between the text sequence length at play here? I've got some preliminary test runs suggesting this is the case. Increasing the text_seq_len causes loss to converge...

@janEbert I think they may have been trying to underfit the data for some reason because of issues which are perhaps more apparent at the scale OpenAI was operating at....

The memory constraints make this incredibly difficult to do on _most_ hardware. As such, 256 pixels is the largest image size you're likely to get working. We don't provide a...

> I tried training with COCO dataset, and the results aren't good. Here is one sample image which I specify as 'green cat'. Anyone else with better results to share?...

This is an early output (2 epochs) from the new code that removes the normalization from train_dalle.py. Was that the necessary fix @lucidrains ? ``` DEPTH = 6 BATCH_SIZE =...

> Hi @afiaka87, Amazing results! Can you share more details about your configurations? such as the dataset, learning rate, lr scheduler, number of text and image (8192, right?) tokens? Thanks....

Edit: You can find the whole training session here: edit: edit: err here: https://wandb.ai/afiaka87/dalle-pytorch-openai-samples/reports/Training-on-OpenAI-DALL-E-Generated-Images--Vmlldzo1MTk2MjQ?accessToken=89u5e10c2oag5mlv46xm2sz6orkyqdlwjrsj8vd95oz8ke3ez6v8v2fh07klk6j1 I'm starting over because there have been updates to the main branch. Original post: "a professional...