talmolab
Render Video > Crop Size: Half results in cv2.error
Just wanted to give you a heads up that some of the video rendering options are causing errors.
It seems I can only successfully render a video clip when I don't choose to crop the output. When I try to crop by half, I get this error message in the Anaconda Prompt:
Writing video with 11259 frame images...
INFO:sleap.io.visuals:Chunks: 176, chunk size: 64
Exception in thread Thread-8:
Traceback (most recent call last):
File "c:\users\vlcor\.conda\envs\sleap_env\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "c:\users\vlcor\.conda\envs\sleap_env\lib\site-packages\sleap\io\visuals.py", line 215, in run
self.marker()
File "c:\users\vlcor\.conda\envs\sleap_env\lib\site-packages\sleap\io\visuals.py", line 235, in marker
frame_images=video_frame_images,
File "c:\users\vlcor\.conda\envs\sleap_env\lib\site-packages\sleap\io\visuals.py", line 251, in _mark_images
video_frame=frame_images[i], frame_idx=frame_idx
File "c:\users\vlcor\.conda\envs\sleap_env\lib\site-packages\sleap\io\visuals.py", line 274, in _mark_single_frame
return cv2.addWeighted(overlay, self.alpha, video_frame, 1 - self.alpha, 0)
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'cv::arithm_op'
(and the GUI freezes).
Also, when I do make a full sized video, and I try to open it in VLC, it says it has to "Fix AVI". It does get fixed and then I can view it, so it's not an urgent issue. Just wanted to let you know about these errors!
Thanks!!
Hi @vcorbit,
We'll fix the crop by half bug, but for the second, what do you see in the render clip windows about the backend?
If it doesn't say ffmpeg, do you mind trying to conda install ffmpeg in the SLEAP environment and seeing if you can render as MP4 and see if it's still corrupted?
Thanks!
Talmo
Ooh, @talmo , that worked well! It did say something about MP4s being encoded with ffmpeg, so when I made the filename I just explicitly wrote ".mp4" and now it's better. I had tried that earlier but that was also when I was trying the half-crop and getting that error, which I had wrongly attributed to putting the extension there.
Thanks so much!
Problem Analysis
P1: Add arrays of different sizes with cv2.addWeighted
In VideoMarkerThread._mark_single_frame, we attempt to cv2.addWeighted two np.ndarrays of different sizes which is not allowed per the docs.
P2: Return a tuple instead of expected np.ndarray
Sometimes, we attempt to add a np.ndarray with a tuple since we occasionally return a tuple from VideoMarkerThread._plot_instances_cv if there are no labeled frames in the current video and the user elected to crop the video.
P3: Not all images are cropped to same size
Although Instances.centroids computes a np.nanmedian, some the NaNs are retruned when taking the np.median(centroids) in VideoMarkerThread._get_instance_center.
P4: GUI freezes when we run into an Exception
There was no try:... except:... block to handle errors in while connection to skvideo.io.FFmpegWriter open.
