Tianhong Li

It only uses training set.

Change the dataset here to your customized dataset implementation https://github.com/LTH14/mage/blob/main/main_pretrain.py#L122

Yes, they are deprecated here.

We use a BertEmb which include its own positional embedding https://github.com/LTH14/mage/blob/main/models_mage.py#L96

We use categorical sampling to generate different images. When a blank image is input to the encoder, [C0] as well as 256 [M] is input to the encoder (during generation).

We do not include a class-conditional image in this repo. The 1024+1000+1 token_emb is actually deprecated. When doing class conditional generation, we use another token_embedding which embeds the class label...

I just updated the README to include pre-trained MAGE-C checkpoints converted from JAX. We are still wrapping up the PyTorch code for the contrastive loss. We will endeavor to release...

Please refer to this [paper](https://arxiv.org/abs/2210.16870) for that comparison. We also have the comparison between MAGE and MAGE-C in Table 2 and Table 13 in our [paper](https://arxiv.org/pdf/2211.09117.pdf).

We haven't tried an encoder-only structure like BEiT. The reason why we chose the MAE-style enc-dec structure is simply that they were the sota method at that time. Also, an...

The pre-trained model we release can perform unconditional image generation on ImageNet. If you want a model for unconditional generation on your own dataset, you need to pre-train MAGE on...