Where are temp frames kept?

[Ainaemaet]

trafficstars

Hello, a nice little project you have here. I'm getting some great results. Just wondering if you can please tell me where are temp frames saved to before putting them together?

A couple times now I've run into an issue (self induced) where I am running a generation that is just at the edge of what my 24Gb vram will allow, and after the loading and generating it hangs forever (seemingly, for hours at least) at the final step. Would be nice to know where things are going before being fed into ffmpeg so I can stitch them together manually!

[ExponentialML]

@Ainaemaet There is a bug with the VAE where I ran into this problem. You can use this mini batching-esque code to fix it at: https://github.com/Vchitect/SEINE/blob/70308b2683a333121476b63274cdbe0ecb81b708/sample_scripts/with_mask_sample.py#L168

video_frames = []

for i in range(video_clip.shape[0]):
    frame = vae.decode(video_clip[[i]] / 0.18215).sample # [16, 3, 256, 256]
    video_frames.append(frame)
  
video_clip = torch.cat(video_frames, dim=0)

As for the temporary frames, everything is done in memory and not saved to disk until inference is complete.

[Ainaemaet]

Thanks for getting back to me so quickly, I will have to wait and try it out in the morning (it's 3am here atm) but I'll report how that works.