zyx

Wukong token-wise 交互的模型图文相似性区别度较小？

1

作者您好，我用同样的图文输入，试验了一下wukong_vit_b和wukong_vit_b_g两个模型，相似度结果是wukong_vit_b：[0.0855, 0.0848, 0.0854, 0.0881]，wukong_vit_b_g：[0.0521, 0.0429, 0.0910, 0.1226]，gt是index=3的位置，结果未作softmax。 wukong_vit_b的结果在小数点后3位才能体现出来，而wukong_vit_b_g区分性反而好一些，这样看token-wise好像并没有起到很好的效果。请问，你们那边实验的也是这样的吗？

Traceback (most recent call last): File "main.py", line 851, in trainer.fit(model, data) File "/opt/conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 553, in fit self._run(model) File "/opt/conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 918, in _run self._dispatch() File "/opt/conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 986,...

FID/IS: how can i calculate the conditional models' FID/IS score?

5

I only find the unconditional models fid/is calculate codes in sample_diffusion.py. how can i calculate the conditional models' FID/IS score?