how to encode and output file from anime4k？

[lsl330]

I have no problem playing anime4k video via mpv, and I have no problem playing it via command line, but I want to output the video to 1080p, how should I achieve that?

command mpv input.mkv --no-sub --vo=gpu --glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_Restore_CNN_M.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" it work and play as well

but command mpv input.mkv --no-sub --vo=gpu --glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_Restore_CNN_M.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale=1440:1080 --o=output.mkv No effect， how can i set the Resolution to 1080p and output the anime4k video？

[hooke007]

no meaning of using vo=gpu whiling encoding.... To be clear, do not use it. vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

[lsl330]

no meaning of using vo=gpu whiling encoding.... To be clear, do not use it. vf=gpu="w=1920:h=1080"

https://mpv.io/manual/master/#video-filters-gpu

thank you for reply,but when i use vf=gpu="w=1920:h=1080" it have an error, Option vf: gpu doesn't exist. Error parsing option vf (option parameter could not be parsed) Setting commandline option --vf=gpu=w=1440:h=1080 failed.

[hooke007]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. https://github.com/mpv-player/mpv/issues/9354

[lsl330]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

thank you, it seems that I can only wait until later to add support for Win

[haasn]

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

[lsl330]

If your ffmpeg is built with the corresponding options, you can use the vf_libplacebo filter in ffmpeg to apply shaders, including Anime4K, while transcoding.

more detial？ 标语言英语（美式） Can you please give me the full command line?

[aypro]

I compiled the latest ffmpeg commit with libplacebo filter, but i'm getting this error when trying to add it to my command ffmpeg -y -i clip.avs -vf libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl -vcodec libx264 -an out.mp4

[avisynth @ 000001c5f71da4c0] Stream #0: not enough frames to estimate rate; consider increasing probesize Guessed Channel Layout for Input Stream #0.1 : stereo Input #0, avisynth, from 'clip.avs': Duration: 00:00:05.84, start: 0.000000, bitrate: 1 kb/s Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le, 1920x1080, 23.98 fps, 23.98 tbr, 23.98 tbn Stream #0:1: Audio: pcm_s32le, 48000 Hz, stereo, s32, 3072 kb/s Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264)) Press [q] to stop, [?] for help Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:0 Conversion failed!

[haasn]

@lsl330 @aypro it's implemented as a vulkan filter so you have to upload the frame to vulkan first:

ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT

[aypro]

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

[haasn]

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

[xiaowu2001]

I have problem when I use bat to upscale video. However something was wrong in cmd: Trailing option(s) found in the command: may be ignored. Device creation failed: -12. Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory Error parsing global options: Cannot allocate memory

[xiaowu2001]

Thank you @haasn , this is what ended up working for me, just had to explicitly add the format on both ends, also i don't know if this is the correct way to apply multiple shaders ffmpeg -y -i IN.MKV -init_hw_device vulkan -vf format=yuv420p,hwupload,libplacebo=custom_shader_path=Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=Anime4K_Restore_CNN_Moderate_VL.glsl,libplacebo=custom_shader_path=Anime4K_Darken_HQ.glsl,libplacebo=custom_shader_path=Anime4K_Thin_HQ.glsl,hwdownload,format=yuv420p -vcodec libx264 -an out.mp4

Can you upscale video by this command successfully? There's something wrong when I use the command. the error in cmd: "Trailing option(s) found in the command: may be ignored. Device creation failed: -12. Failed to set value 'vulkan' for option 'init_hw_device': Cannot allocate memory" I'm not good at ffmpeg, and I'm not sure if it's the problem of command "-init_hw_device vulkan".

[hooke007]

check if your local build support vulkan ffmpeg -init_hw_device list

[xiaowu2001]

check if your local build support vulkan ffmpeg -init_hw_device list

Supported hardware device types: cuda dxva2 qsv d3d11va

[hooke007]

So, obviously no.

[xiaowu2001]

Can I use "cuda" instead of "vulkan"? Or how can I output enlarged video by other way?

So, obviously no.

[xiaowu2001]

That's a bit wasteful, it would be much better to cat all shaders into the same file and load that. (Though I'll probably fix vf_libplacebo to accept multiple shaders soon-ish)

That said, to make it significantly less wasteful, you could add format=rgba (or rgba64le) to the first libplacebo filter and format=yuv420p to the last libplacebo filter. (That way you at least avoid constantly converting between rgb and yuv420p)

My ffmpeg(v5.0) build doesn't support vulkan, how to solve the problem or if there's anyway else to achieve outputting videos from Anime4K? And I also have a question about ffmpeg -i CLIP -init_hw_device vulkan -vf hwupload,libplacebo,hwdownload -vcodec libx264 OUT, "libplacebo" is used to set custom path of filter, right?

[k4yt3x]

With a standard Linux installation of GLSL for MPV, here's the full command I used to render the output. It works like a charm. IIRC this should be using the same shaders as Mode A.

mpv input.mp4 --no-sub --glsl-shaders="~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf=gpu="w=3840:h=2160" --o=output.mkv

Edit: this should be easier to read.

mpv input.mp4 \
    --no-sub \
    --glsl-shaders="\
~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:\
~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" \
    -vf=gpu="w=1216:h=2160" \
    --o=output.mkv

[infinisil]

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

[k4yt3x]

@k4yt3x There's a mistake in your last command, the width should be 3840, not 1216

That's for the specific video I'm processing. Just do the calculations and change it to whatever you need it to be.

[studentnice05]

This requires that the EGL implementation supports off-screen rendering on the default display. (This is the case with Mesa.)

See manual. windows does not support now. #9354

现在在windows上不能使用--vf=gpu的话是不是就无法保存anime4k放大的视频了？Windows在运行下面的命令时候总是会报错：mpv input.mp4 --glsl-shaders="%appdata%/mpv/shaders/Anime4K_Clamp_Highlights.glsl" --vf=gpu -vf scale="1920:1080" --o=output.mkv 报错：Option vf: gpu doesn't exist.

[Hrxn]

it's --vo=gpu

[hooke007]

报错：Option vf: gpu doesn't exist.

Because there is no implement in windows. https://github.com/mpv-player/mpv/issues/9589#issuecomment-991900640

it's --vo=gpu

No, it's vf not vo. It's clear in the doc and previous comments.

[SwayStar123]

With a standard Linux installation of GLSL for MPV, here's the full command I used to render the output. It works like a charm. IIRC this should be using the same shaders as Mode A.

mpv input.mp4 --no-sub --glsl-shaders="~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf=gpu="w=3840:h=2160" --o=output.mkv

Edit: this should be easier to read.

mpv input.mp4 \
    --no-sub \
    --glsl-shaders="\
~/.config/mpv/shaders/Anime4K_Clamp_Highlights.glsl:\
~/.config/mpv/shaders/Anime4K_Restore_CNN_VL.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x2.glsl:\
~/.config/mpv/shaders/Anime4K_AutoDownscalePre_x4.glsl:\
~/.config/mpv/shaders/Anime4K_Upscale_CNN_x2_M.glsl" \
    -vf=gpu="w=1216:h=2160" \
    --o=output.mkv

What am i doing wrong? Im using WSL btw

[mikigal]

I encoded video using this command: mpv input.mp4 --glsl-shader="~~/shaders/Anime4K_Clamp_Highlights.glsl:~~/shaders/Anime4K_Restore_CNN_VL.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl:~~/shaders/Anime4K_AutoDownscalePre_x2.glsl:~~/shaders/Anime4K_AutoDownscalePre_x4.glsl:~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl" -vf scale="2560:1440" -o output.mp4

I got output file 1440p resolution, but it looks shaders are not applied - output file looks same as input. It just scaled file. When I use vf=gpu nothing happens.

Did anyone find way to encode video with Anime4K shaders?

Platform: Windows

[haasn]

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

[mikigal]

Did anyone find way to encode video with Anime4K shaders?

FFmpeg with vf_libplacebo is the recommended way.

Already tried all ways from this thread, still getting errors. Can you share correct parameters for ffmpeg?

[zanechua]

On Windows:

Download ffmpeg build from https://github.com/BtbN/FFmpeg-Builds/releases

./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4

Modify the scale and formats accordingly.

There's something wonky with libplacebo at the moment not liking a particular statement in the glsl filters.

Remove //!SAVE MAIN from all the glsl filters otherwise nothing will happen.

Tested this on Windows 11 with a 3080 Ti.

EDIT:

After a bit more testing, I can now tell that certain filters are being applied, and certain filters are not. There's still a discernable difference in certain files, not exactly sure where the issue is.

[ryandash]

On Windows:

Download ffmpeg build from https://github.com/BtbN/FFmpeg-Builds/releases

./ffmpeg.exe -y -i input.mp4 -init_hw_device vulkan -vf scale=3840:2160,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_Clamp_Highlights.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_VL.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Restore_CNN_M.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x2.glsl,libplacebo=custom_shader_path=shaders/Anime4K_AutoDownscalePre_x4.glsl,libplacebo=custom_shader_path=shaders/Anime4K_Upscale_CNN_x2_M.glsl,hwdownload,format=yuv420p -vcodec libx264 out.mp4

Modify the scale and formats accordingly.

There's something wonky with libplacebo at the moment not liking a particular statement in the glsl filters.

Remove //!SAVE MAIN from all the glsl filters otherwise nothing will happen.

Tested this on Windows 11 with a 3080 Ti.

EDIT:

After a bit more testing, I can now tell that certain filters are being applied, and certain filters are not. There's still a discernable difference in certain files, not exactly sure where the issue is.

This worked pretty well for me with some small tweaks and simplified shaders.

I merged all the shaders for each mode into files using copy filter1.glsl + filter2.glsl + filteretc... "outputFilterName.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4k_ModeB.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_modeC.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeA+A.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Restore_CNN_Soft_VL.glsl + Anime4K_Upscale_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_Soft_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeB+B.glsl"

copy Anime4K_Clamp_Highlights.glsl + Anime4K_Upscale_Denoise_CNN_x2_VL.glsl + Anime4K_AutoDownscalePre_x2.glsl + Anime4K_AutoDownscalePre_x4.glsl + Anime4K_Restore_CNN_M.glsl + Anime4K_Upscale_CNN_x2_M.glsl "Anime4K_ModeC+A.glsl"

I then removed all the //!SAVE MAIN from the merged glsl files

For anyone interested in downloading these merged shaders here they are: shaders.zip

I then used PowerShell to run FFmpeg with the following command

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p10,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p10 -vcodec libx264 "outputVideo.mkv"

The flags=lanczos is what made the biggest difference for me, without it the final output had thicker lines and was not nearly as sharp as it was compared to mpv. https://write.corbpie.com/a-guide-to-upscaling-or-downscaling-video-with-ffmpeg/#:~:text=Scaling%20with%20a%20video%20scaling%20algorithm did a good job explaining the possible flags options.

Original Image:

anime4k mode A filter in MPV:

anime4k mode A filter ffmpeg:

anime4k mode A filter ffmpeg without flags=lanczos:

I do apologize if the images are not full quality because they were compressed to upload to github. I also apologize if the comparison is a little annoying because they are not on the same frame. I was a little lazy.

There may be some differences between the pictures but they are small enough that I did not notice them while watching a short clip. I also use "ewa_lanczossharp" in mpv which is slightly different from lanczos

This is the mpv.conf file I used for the anime4k mode A filter in MPV

profile=gpu-hq
save-position-on-quit=yes
scale=ewa_lanczossharp
cscale=ewa_lanczossharp
alang=jpn,eng
slang=eng,jpn
--sub-blur=0.3
glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_VL.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl"

[Jules-A]

@ryandash I tried that but I just get

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Assertion failed: sem_out.sem, file ../src/vulkan/gpu_tex.c, line 1121

[ryandash]

@ryandash I tried that but I just get

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Assertion failed: sem_out.sem, file ../src/vulkan/gpu_tex.c, line 1121

Try this

./ffmpeg.exe -y -i "inputVideo.mp4" -init_hw_device vulkan -vf scale=3840:2160:flags=lanczos,format=yuv420p,hwupload,libplacebo=custom_shader_path=shaders/Anime4K_ModeA.glsl,hwdownload,format=yuv420p -vcodec libx264 "outputVideo.mkv"

I assume you are using an AMD GPU. I only have an nvidia gpu to test with and there does seem to be known issues with AMD GPUs when converting with ffmpeg on Windows. If there are still issues then you can try https://ubuntu.com/wsl which let's you use Linux on windows. The Linux mesa driver for AMD GPUs seems to be more stable.