video2x
video2x copied to clipboard
k4yt3x
Upscaling `.bik` Video with h264_nvenc Fails
System Info
OS: Gentoo Linux GPU: Nvidia RTX 2080 Ti GPU Driver: 535.104.05
Description
I was trying to upscale cutscene .bik videos for an old game (namely Splinter Cell 1 from Steam), and video2x seems to be throwing an error related to ffmpeg, somehow, and it's unclear to me what exactly the problem is when reading the log (and there also appears to be several things going wrong - such as a missing streams?)...
Ideally, for context, I also need the output to be back in the .bik format but it doesn't seem to be possible with ffmpeg(? in a prior test), so I opted to output it to mkv until I can find a way to convert it back (but that's a bit off-topic from the main issue here).
Log and Config File
video2x.yaml configuration file:
# Name: Video2X Configuration File
# Values here are the default values. Change the value here to
# save the default value permanently.
# Items commented out are parameters irrelevant to this context
# or parameters handled by Video2X internally.
waifu2x_caffe:
path: '/usr/bin/waifu2x-caffe-cui'
tta: 0 # <0|1> 8x slower and slightly high quality
gpu: 0 # gpu device no
batch_size: 1 # input batch size
crop_h: null # input image split size(height)
crop_w: null # input image split size(width)
crop_size: 128 # input image split size
output_depth: 8 # output image chanel depth bit
output_quality: -1 # output image quality
process: gpu # <cpu|gpu|cudnn> process mode
model_dir: null # path to custom model directory (don't append last / )
#scale_height: 0 # custom scale height (specifying this will overwrite scale_ratio)
#scale_width: 0 # custom scale width (specifying this will overwrite scale_ratio)
#scale_ratio: null # custom scale ratio
noise_level: 3 # <0|1|2|3> noise reduction level
mode: noise_scale # <noise|scale|noise_scale|auto_scale> image processing mode
output_extention: null # extension to output image file when output_path is (auto) or input_path is folder
input_extention_list: null # extension to input image file when input_path is folder
#output_path: null # path to output image file (when input_path is folder, output_path must be folder)
#input_path: null # (required) path to input image file
waifu2x_converter_cpp:
path: '/usr/bin/waifu2x-converter-cpp'
#list-supported-formats: null # dump currently supported format list
#list-opencv-formats: null # (deprecated. Use --list-supported-formats) dump opencv supported format list
#list-processor # dump processor list
#output-format: null # The format used when running in recursive/folder mode
png-compression: 5 # Set PNG compression level (0-9), 9 = Max compression (slowest & smallest)
image-quality: -1 # JPEG & WebP Compression quality (-1-101, 0 being smallest size and lowest quality, -1 being default), use 101 for lossless WebP
block-size: 0 # block size
disable-gpu: false # disable GPU
force-OpenCL: false # force to use OpenCL on Intel Platform
processor: -1 # set target processor (-1 uses default device)
jobs: 0 # number of threads launching at the same time
model-dir: null # path to custom model directory (don't append last / ) default: models_rgb
#scale-ratio: 2.0 # custom scale ratio
noise-level: 1 # <0|1|2|3> noise reduction level
mode: noise-scale # <noise|scale|noise-scale> image processing mode
log-level: 1 # <0|1|2|3|4> Set log level
silent: true # Enable silent mode. (same as --log-level 1)
tta: 0 # Enable Test-Time Augmentation mode. (0 or 1)
#generate-subdir: 0 # Generate sub folder when recursive directory is enabled.
#auto-naming: 0 # Add postfix to output name when output path is not specified.
#recursive-directory: 0 # Search recursively through directories to find more images to process.
#output: null # path to output image file or directory (you should use the full path)
#input: null # (required) path to input image file or directory (you should use the full path)
waifu2x_ncnn_vulkan:
path: '/usr/bin/waifu2x-ncnn-vulkan'
v: null # verbose output
#i: null # input-path: input image path (jpg/png/webp) or directory
#o: null # output-path: output image path (jpg/png/webp) or directory
'n': 3 # noise-level: denoise level (-1/0/1/2/3, default=0)
s: 2 # scale: upscale ratio (1/2, default=2)
t: 400 # tile-size: tile size (>=32, default=400)
m: null # model-path: waifu2x model path (default=models-cunet)
g: 0 # gpu-id: gpu device to use (default=0)
j: '1:2:2' # thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
x: false # enable tta mode
#f: png # output image format (jpg/png/webp, default=ext/png)
srmd_ncnn_vulkan:
path: '/usr/bin/srmd-ncnn-vulkan'
v: null # verbose output
#i: null # input-path: input image path (jpg/png) or directory
#o: null # output-path: output image path (png) or directory
'n': 3 # noise-level: denoise level (-1/0/1/2/3/4/5/6/7/8/9/10, default=3)
s: 2 # upscale ratio (2/3/4, default=2)
t: 400 # tile-size: tile size (>=32, default=400)
m: null # srmd model path (default=models-srmd)
g: 0 # gpu device to use (default=0)
j: '1:2:2' # thread count for load/proc/save (default=1:2:2)
x: false # enable tta mode
#f: png # output image format (jpg/png/webp, default=ext/png)
realsr_ncnn_vulkan:
path: '/usr/bin/realsr-ncnn-vulkan'
v: null # verbose output
#i: null # input-path: input image path (jpg/png) or directory
#o: null # output-path: output image path (png) or directory
s: 4 # upscale ratio (4, default=4)
t: 0 # tile size (>=32/0=auto, default=0)
m: null # realsr model path (default=models-DF2K)
g: 0 # gpu device to use (default=0)
j: '1:2:2' # thread count for load/proc/save (default=1:2:2)
x: false # enable tta mode
#f: png # output image format (jpg/png/webp, default=ext/png)
anime4kcpp:
path: '/usr/bin/Anime4KCPP_CLI'
#input: null # File for loading (string [=./pic/p1.png])
#output: null # File for outputting (string [=output.png])
passes: 2 # Passes for processing (int [=2])
pushColorCount: 2 # Limit the number of color pushes (int [=2])
strengthColor: 0.3 # Strength for pushing color,range 0 to 1,higher for thinner (double [=0.3])
strengthGradient: 1.0 # Strength for pushing gradient,range 0 to 1,higher for sharper (double [=1])
zoomFactor: 2.0 # zoom factor for resizing (double [=2])
threads: 16 # Threads count for video processing (unsigned int [=16])
fastMode: false # Faster but maybe low quality
videoMode: false # Video process
preview: null # Preview image
preprocessing: false # Enable pre processing
postprocessing: false # Enable post processing
preFilters: 4 # Enhancement filter, only working when preProcessing is true,there are 5 options by binary:Median blur=0000001, Mean blur=0000010, CAS Sharpening=0000100, Gaussian blur weak=0001000, Gaussian blur=0010000, Bilateral filter=0100000, Bilateral filter faster=1000000, you can freely combine them, eg: Gaussian blur weak + Bilateral filter = 0001000 | 0100000 = 0101000 = 40(D) (unsigned int [=4])
postFilters: 40 # Enhancement filter, only working when postProcessing is true,there are 5 options by binary:Median blur=0000001, Mean blur=0000010, CAS Sharpening=0000100, Gaussian blur weak=0001000, Gaussian blur=0010000, Bilateral filter=0100000, Bilateral filter faster=1000000, you can freely combine them, eg: Gaussian blur weak + Bilateral filter = 0001000 | 0100000 = 0101000 = 40(D), so you can put 40 to enable Gaussian blur weak and Bilateral filter, which also is what I recommend for image that < 1080P, 48 for image that >= 1080P, and for performance I recommend to use 72 for video that < 1080P, 80 for video that >=1080P (unsigned int [=40])
GPUMode: false # Enable GPU acceleration
CNNMode: false # Enable ACNet
HDN: false # Enable HDN mode for ACNet
HDNLevel: 1 # Set HDN level (int [=1])
listGPUs: null # list GPUs
platformID: 0 # Specify the platform ID (unsigned int [=0])
deviceID: 0 # Specify the device ID (unsigned int [=0])
codec: avc1 # Specify the codec for encoding from mp4v(recommended in Windows), dxva(for Windows), avc1(H264, recommended in Linux), vp09(very slow), hevc(not support in Windowds), av01(not support in Windowds) (string [=mp4v])
forceFps: 0 # Set output video fps to the specifying number, 0 to disable (float [=0])
disableProgress: false # disable progress display
webVideo: null # process the video from URL
alpha: false # preserve the Alpha channel for transparent image
ffmpeg:
ffmpeg_path: '/usr/bin/'
intermediate_file_name: 'intermediate.mkv'
# Step 1: Frame Extraction
# extract all frames from original input into temporary directory
extract_frames:
'-hwaccel': auto # automatically select hardware acceleration method
'-y': true
input_options: {} # empty dict, expand if necessary
output_options:
'-pix_fmt': rgb24 # extracted frames pixel format
'-qscale:v': null # output image quality control for JPEG (1-31, 1 being highest)
# Step 2: Video Assembly
# assemble all frames back into a video with only a video track
assemble_video:
'-hwaccel': auto # automatically select hardware acceleration method
'-y': true
input_options:
'-f': image2 # force image2 format
output_options:
'-vcodec': h264_nvenc # video codec
'-pix_fmt': 'yuv420p' # overwrite default pixel format
'-crf': 17 # H.264 Constant Rate Factor
'-b:v': null # target average bitrate
'-vf': 'pad=ceil(iw/2)*2:ceil(ih/2)*2' # ensure output is divisible by 2, recommended for libx264
'-tune': 'animation' # encoding tuning film/animation/grain/stillimage/fastdecode/zerolatency/psnr/ssim
# Step 3: Streams Migration
# migrate audio and subtitle streams from original
# video into the upscaled video
migrate_streams:
'-hwaccel': auto # automatically select hardware acceleration method
'-y': true
input_options: {} # empty dict, expand if necessary
output_options:
'-map':
- '0:v?' # copy video streams
- '1:a?' # copy audio streams
- '1:s?' # copy subtitle streams
- '1:d?' # copy data streams
- '1:t?' # copy fonts
'-c': copy # copy codec for all streams
# '-vf': 'minterpolate=''fps=60''' # minterpolate frame interpolation
'-map_metadata': 0 # copy known metadata tags
# '-movflags': 'use_metadata_tags' # copy custom/arbitrary metadata tags
'-pix_fmt': null
'-metadata': 'comment=Upscaled by Video2X'
gifski:
gifski_path: '/usr/bin/gifski'
# output: null # Destination file to write to
# fps: 20 # Animation frames per second (for PNG frames only) [default: 20]
fast: false # 3 times faster encoding, but 10% lower quality and bigger file
quality: 100 # Lower quality may give smaller file
#width: null # Maximum width
#height: null # Maximum height (if width is also set)
once: false # Do not loop the GIF
nosort: false # Use files exactly in the order given, rather than sorted
quiet: false # Do not show a progress bar
video2x:
video2x_cache_directory: /home/neko-san/.cache/video2x # default: %TEMP%\video2x, directory where cache files are stored, will be deleted if preserve_frames is not set to true
extracted_frame_format: png # png/jpg intermediate file format used for extracted frames during video processing
output_file_name_format_string: "{original_file_name}_upscaled{extension}" # format string to use for generating output file names
image_output_extension: .png # image output extension during batch processing
video_output_extension: .mkv # video output extension during batch processing
preserve_frames: false # if set to true, the cache directory won't be cleaned upon task completion
Did you try to run the command by itself?
/usr/bin/ffmpeg -r 29.419795221843003 -hwaccel auto -y -f image2 -i /home/neko-san/.cache/video2x/tmptp_c75mh/extracted_%d.png -vcodec h264_nvenc -pix_fmt yuv420p -crf 17 -vf pad=ceil(iw/2)*2:ceil(ih/2)*2 -tune animation /home/neko-san/.cache/video2x/tmptp_c75mh/intermediate.mkv
There are two weird errors here
Codec AVOption crf (Select the quality for constant quality mode) specified for output file #0 (/home/neko-san/.cache/video2x/tmptp_c75mh/intermediate.mkv) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
Stream #0:0 -> #0:0 (png (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264_nvenc @ 0x561d9e27d440] [Eval @ 0x7ffc87379a78] Undefined constant or missing '(' in 'animation'
[h264_nvenc @ 0x561d9e27d440] Unable to parse option value "animation"
[h264_nvenc @ 0x561d9e27d440] Error setting option tune to value animation.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
And last but not least, everybody uses rad tools to encode to bink.
Looks like you're using an old version of Video2X on Gentoo, which I would not recommend albeit it may not have anything to do with the error. Consider Docker or Colab.
Follow what @mirh said to see if FFmpeg can work with the format at all. If not, convert to another common format before processing it.
