Open-Sora icon indicating copy to clipboard operation
Open-Sora copied to clipboard

为什么要在推理时复制一份latent进行推理,可以移除吗?

Open yusijin02 opened this issue 1 year ago • 3 comments

在进行推理时: 为什么要先做 https://github.com/hpcaitech/Open-Sora/blob/a37a189482a4cd1c7892aa06881e539cbf8078ce/opensora/schedulers/iddpm/init.py#L66 再做 https://github.com/hpcaitech/Open-Sora/blob/a37a189482a4cd1c7892aa06881e539cbf8078ce/opensora/schedulers/iddpm/init.py#L83 手动移除有什么弊端吗?

yusijin02 avatar Mar 27 '24 06:03 yusijin02

我留意到y_null的引入: https://github.com/hpcaitech/Open-Sora/blob/a37a189482a4cd1c7892aa06881e539cbf8078ce/opensora/schedulers/iddpm/init.py#L69 但是在样本级别的拼接为什么会影响输出样本?

yusijin02 avatar Mar 27 '24 06:03 yusijin02

我留意到y_null的引入:

https://github.com/hpcaitech/Open-Sora/blob/a37a189482a4cd1c7892aa06881e539cbf8078ce/opensora/schedulers/iddpm/init.py#L69

但是在样本级别的拼接为什么会影响输出样本?

涉及到classifier-free guidance(cfg)的相关知识,你可以参考下链接中的forward_with_cfg函数,包含了样本间输出的融合

ictzyqq avatar Apr 02 '24 03:04 ictzyqq

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] avatar Apr 17 '24 01:04 github-actions[bot]

This issue was closed because it has been inactive for 7 days since being marked as stale.

github-actions[bot] avatar Apr 24 '24 01:04 github-actions[bot]