Ruoxi
Ruoxi
An experimental implementation of this is available at https://github.com/eliphatfs/Real-ESRGAN/blob/master/inference_realesrgan_video_fast.py Reduced IO overhead and implemented batch inference. On my 3080 LP it is able to boost 1080p to 4k upscaling from...
I did a few more tweaks and the frame rate is now 4.1 to 4.2 fps on my 3080 LP. For my testing video there is still 10% time doing...
> @eliphatfs in order to implement your '_fast.py` script, may I simply drag and drop such into the vanilla Real-ESRGAN repo? I've attempted such and am running into this error:...
P.S. The script does not yet support alpha channels or gray-scale. Only RGB videos are supported (I think this should be the case of 99% modern videos). `extract_frame_first` may be...
> I tried ViT-L/14. You have to just change it in inference code and feature extractor code. For example `parse_coco.py`: > > ``` > parser.add_argument('--clip_model_type', default="ViT-L/14", choices=('RN50', 'RN101', 'RN50x4', 'ViT-B/32',...
I have a basic alternative solution using the `trimesh` library. Also, if you have this same concern as me: vertices generated by marching cubes are very dense, and vertex color...
> This is pretty simple actually, just use "with accelerator.accumulate(model1), accelerator.accumulate(model2): " this is the mechanism of "with", the following code will be in this two contexts, so just simply...
TL; DR: don't do gradient accumulation with multiple models. Wrap them in a wrapper model and do accelerator stuff with it. Move relevant forward logic inside the wrapper model. Edit:...
Basically, if you have this in your main training loop: ```python states = text_encoder(input_ids) pred = unet(noisy_latents, states, timesteps) loss = F.mse_loss(pred, targets) # now loss.backward() will corrupt gradients if...