使用onediff后Comfyui Ip-adapter没生效

[HydrogenQAQ]

Describe the bug

A clear and concise description of what the bug is.

同时跑了不加onediff和加了onediff的，seed一样，模型一样，加了onediff的ip-adapter并没有生效

Your environment

OS

OneDiff git commit id

onediff：b89dd4980aeab85b8f9c comfyui：f81dbe26e2e363c28ad043 ip-adapter：d87b7ce1c1b3c22

OneFlow version info

Run python -m oneflow --doctor and paste it here.

version: 0.9.1.dev20240124+cu121 git_commit: 6458a12 cmake_build_type: Release rdma: True mlir: True enterprise: False

How To Reproduce

Steps to reproduce the behavior(code or script): test.json

The complete error message

Additional context

Add any other context about the problem here.