QCTools (Windows) crash on analysis of long video
QCTools on Windows crash when I analyse longer video (around 30-40 minutes). This is a video track of opatom pattern, xdcam. QCTools analyses around 25minutes and then crashes when I analyse it with qcli - I see progress reaching 100% but report is generated for first 8 minutes only
Hi @amietus, can you run ffmpeg -i your_file.mxf on the file that you are trying to analyze. That will help us try to recreate the error.
Hello sure, here is the output:
ffmpeg.exe -i h:\DNX33_145V013397006D.mxf ffmpeg version N-89881-g1948b76a1b Copyright (c) 2000-2018 the FFmpeg developers built with gcc 7.2.0 (GCC) configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth libavutil 56. 7.100 / 56. 7.100 libavcodec 58. 9.100 / 58. 9.100 libavformat 58. 5.101 / 58. 5.101 libavdevice 58. 0.101 / 58. 0.101 libavfilter 7. 11.101 / 7. 11.101 libswscale 5. 0.101 / 5. 0.101 libswresample 3. 0.101 / 3. 0.101 libpostproc 55. 0.100 / 55. 0.100 [mxf @ 000001a1aeb19780] could not resolve source track sequence strong ref Input #0, mxf, from 'h:\DNX33_145V013397006D.mxf': Metadata: product_name : vsWrapper uid : 8460f229-e67d-f64d-8295-12912ebe773a generation_uid : b6bcfcab-70ff-7331-9302-42b169de11d2 application_platform: AAFSDK (Linux) modification_date: 2020-12-07T13:56:00.000000Z product_uid : 00000000-0000-0000-0000-000000000000 product_version : company_name : AVID material_package_umid: 0x060A2B340101010101010F00130000007E4D26BE33974F5B84FA6E8F50D69EA1 material_package_name: DNX33_1456_222_201207-1h.1.6 comment_Tape : VE comment_TapeID : DNX33_145BD72VfZ3VK9Kgk-PMsfpFN comment__WG_TRANSFER_TYPE: 0 Duration: 01:04:01.12, start: 0.000000, bitrate: 121243 kb/s Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/unknown, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc Metadata: file_package_umid: 0x060A2B340101010101010F0013000000CDA79F9436E700A57F6336370D1267E9 reel_umid : 0x060A2B340101010101010F0013000000245A3D25F66D4A9CB8A8F6FB7407C550 reel_name : DNX33_1456_222_201207-1h.1.6 Stream #0:1: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28 data_type : audio Stream #0:2: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5292E86B1B4774463 data_type : audio Stream #0:3: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A53393EEE86DD05974 data_type : audio Stream #0:4: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5F0EA93FBD96F5A2B data_type : audio Stream #0:5: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5D66F47906254A3C9 data_type : audio Stream #0:6: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A56A483D979730EE26 data_type : audio Stream #0:7: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5950A5974D80DEC1C data_type : audio Stream #0:8: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5EFA271DA04DEAA5D data_type : audio At least one output file must be specified
Regards Aleksandra
On Thu, 17 Dec 2020 at 17:20, Dave Rice [email protected] wrote:
Hi @amietus https://github.com/amietus, can you run ffmpeg -i your_file.mxf on the file that you are trying to analyze. That will help us try to recreate the error.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-747545038, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSLCSNHKDIAKZCLOZNTSVIVU7ANCNFSM4UR6XLPA .
ffmpeg version N-89881-g1948b76a1b Copyright (c) 2000-2018 the FFmpeg
libavutil 56. 7.100 / 56. 7.100 libavcodec 58. 9.100 / 58. 9.100 libavformat 58. 5.101 / 58. 5.101 libavdevice 58. 0.101 / 58. 0.101 libavfilter 7. 11.101 / 7. 11.101 libswscale 5. 0.101 / 5. 0.101 libswresample 3. 0.101 / 3. 0.101 libpostproc 55. 0.100 / 55. 0.100
This is quite old (in IT terms), but I don’t know if that’s related.
newer one: h:\ffmpeg-N-100421-g1eb751955e-win64-gpl\bin>ffmpeg.exe -i h:\DNX33_145V013397006D.mxf ffmpeg version N-100421-g1eb751955e Copyright (c) 2000-2020 the FFmpeg developers built with gcc 9.3-win32 (GCC) 20200320 configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-zlib --enable-libxml2 --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-opencl --enable-libvmaf --disable-vulkan --enable-libvorbis --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --enable-ffnvcodec --enable-cuda-llvm --disable-libglslang --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-libs=-lgomp libavutil 56. 62.100 / 56. 62.100 libavcodec 58.115.102 / 58.115.102 libavformat 58. 65.100 / 58. 65.100 libavdevice 58. 11.103 / 58. 11.103 libavfilter 7. 93.100 / 7. 93.100 libswscale 5. 8.100 / 5. 8.100 libswresample 3. 8.100 / 3. 8.100 libpostproc 55. 8.100 / 55. 8.100 [mxf @ 000002b6e70044c0] could not resolve source track sequence strong ref Input #0, mxf, from 'h:\DNX33_145V013397006D.mxf': Metadata: operational_pattern_ul: 060e2b34.04010102.0d010201.10020000 product_name : vsWrapper uid : 8460f229-e67d-f64d-8295-12912ebe773a generation_uid : b6bcfcab-70ff-7331-9302-42b169de11d2 application_platform: AAFSDK (Linux) modification_date: 2020-12-07T13:56:00.000000Z product_uid : 00000000-0000-0000-0000-000000000000 product_version : company_name : AVID material_package_umid: 0x060A2B340101010101010F00130000007E4D26BE33974F5B84FA6E8F50D69EA1 material_package_name: DNX33_1456_222_201207-1h.1.6 comment_Tape : VE comment_TapeID : DNX33_145BD72VfZ3VK9Kgk-PMsfpFN comment__WG_TRANSFER_TYPE: 0 Duration: 01:04:01.12, start: 0.000000, bitrate: 121243 kb/s Stream #0:0: Video: dnxhd (DNXHD), yuv422p(tv, bt709/unknown/bt709, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc Metadata: file_package_umid: 0x060A2B340101010101010F0013000000CDA79F9436E700A57F6336370D1267E9 reel_umid : 0x060A2B340101010101010F0013000000245A3D25F66D4A9CB8A8F6FB7407C550 reel_name : DNX33_1456_222_201207-1h.1.6 Stream #0:1: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28 data_type : audio Stream #0:2: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5292E86B1B4774463 data_type : audio Stream #0:3: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A53393EEE86DD05974 data_type : audio Stream #0:4: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5F0EA93FBD96F5A2B data_type : audio Stream #0:5: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5D66F47906254A3C9 data_type : audio Stream #0:6: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A56A483D979730EE26 data_type : audio Stream #0:7: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5950A5974D80DEC1C data_type : audio Stream #0:8: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5EFA271DA04DEAA5D data_type : audio At least one output file must be specified
On Fri, 18 Dec 2020 at 14:02, רטו [email protected] wrote:
ffmpeg version N-89881-g1948b76a1b Copyright (c) 2000-2018 the FFmpeg
libavutil 56. 7.100 / 56. 7.100 libavcodec 58. 9.100 / 58. 9.100 libavformat 58. 5.101 / 58. 5.101 libavdevice 58. 0.101 / 58. 0.101 libavfilter 7. 11.101 / 7. 11.101 libswscale 5. 0.101 / 5. 0.101 libswresample 3. 0.101 / 3. 0.101 libpostproc 55. 0.100 / 55. 0.100
This is quite old (in IT terms), but I don’t know if that’s related.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748072641, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSP5T6KIYD5TJC3TTDLSVNHGZANCNFSM4UR6XLPA .
newer one:
Thank you!
Do you want a sample clip? Bad thing is that it is about 60GB big, but if there is some ftp where I can upload it would be great. Aleksandra
On Fri, 18 Dec 2020 at 14:18, רטו [email protected] wrote:
newer one:
Thank you!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748079506, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSMLOJQXBOQO6ZMVPWLSVNJCJANCNFSM4UR6XLPA .
No, thank you, I don’t think this is needed. I’ll delve into the issue later today using a similar dummy file, which should be sufficient to reproduce and analyse.
BTW: QCTools 1.2 has been released a few hours ago.
Hi @amietus, I'm curious if the order of the non-av tracks is a factor. Could you please test like this:
ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -map 0:a -c copy h:\DNX33_145V013397006D_only_av.mxf
that command will make a version of your file but exclude the data tracks. Then see if the behavior with DNX33_145V013397006D_only_av.mxf in QCTools is the same. Please use QCTools 1.2 for the test.
it works perfectly well for short clips of this format, like 5-7 minutes. but bigger ones are problematic. let me know if you need help / tests :)
Best!
On Fri, 18 Dec 2020 at 14:36, רטו [email protected] wrote:
No, thank you, I don’t think this is needed. I’ll delve into the issue later today using a similar dummy file, which should be sufficient to reproduce and analyse.
BTW: QCTools 1.2 has been released a few hours ago.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748087869, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSOVQIATHBISZPXA3EDSVNLGTANCNFSM4UR6XLPA .
Hi, Your command will not do the conversion:
ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -map 0:a -c copy h:\DNX33_145V013397006D_only_av.mxf
h:\ffmpeg-N-100421-g1eb751955e-win64-gpl\bin>ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -map 0:a -c copy h:\DNX33_145V013397006D_only_av.mxfclear ffmpeg version N-100421-g1eb751955e Copyright (c) 2000-2020 the FFmpeg developers built with gcc 9.3-win32 (GCC) 20200320 configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-zlib --enable-libxml2 --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-opencl --enable-libvmaf --disable-vulkan --enable-libvorbis --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --enable-ffnvcodec --enable-cuda-llvm --disable-libglslang --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-libs=-lgomp libavutil 56. 62.100 / 56. 62.100 libavcodec 58.115.102 / 58.115.102 libavformat 58. 65.100 / 58. 65.100 libavdevice 58. 11.103 / 58. 11.103 libavfilter 7. 93.100 / 7. 93.100 libswscale 5. 8.100 / 5. 8.100 libswresample 3. 8.100 / 3. 8.100 libpostproc 55. 8.100 / 55. 8.100 [mxf @ 0000022a86175700] could not resolve source track sequence strong ref Input #0, mxf, from 'h:\DNX33_145V013397006D.mxf': Metadata: operational_pattern_ul: 060e2b34.04010102.0d010201.10020000 product_name : vsWrapper uid : 8460f229-e67d-f64d-8295-12912ebe773a generation_uid : b6bcfcab-70ff-7331-9302-42b169de11d2 application_platform: AAFSDK (Linux) modification_date: 2020-12-07T13:56:00.000000Z product_uid : 00000000-0000-0000-0000-000000000000 product_version : company_name : AVID material_package_umid: 0x060A2B340101010101010F00130000007E4D26BE33974F5B84FA6E8F50D69EA1 material_package_name: DNX33_1456_222_201207-1h.1.6 comment_Tape : VE comment_TapeID : DNX33_145BD72VfZ3VK9Kgk-PMsfpFN comment__WG_TRANSFER_TYPE: 0 Duration: 01:04:01.12, start: 0.000000, bitrate: 121243 kb/s Stream #0:0: Video: dnxhd (DNXHD), yuv422p(tv, bt709/unknown/bt709, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc Metadata: file_package_umid: 0x060A2B340101010101010F0013000000CDA79F9436E700A57F6336370D1267E9 reel_umid : 0x060A2B340101010101010F0013000000245A3D25F66D4A9CB8A8F6FB7407C550 reel_name : DNX33_1456_222_201207-1h.1.6 Stream #0:1: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28 data_type : audio Stream #0:2: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5292E86B1B4774463 data_type : audio Stream #0:3: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A53393EEE86DD05974 data_type : audio Stream #0:4: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5F0EA93FBD96F5A2B data_type : audio Stream #0:5: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5D66F47906254A3C9 data_type : audio Stream #0:6: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A56A483D979730EE26 data_type : audio Stream #0:7: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5950A5974D80DEC1C data_type : audio Stream #0:8: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D1A79F9436E700A5EFA271DA04DEAA5D data_type : audio Stream map '0:a' matches no streams. To ignore this, add a trailing '?' to the map.
The file is only video track of opatom pattern. So if there are any audios or data track, they are saved in separate mxf files. this is pure video file
On Fri, 18 Dec 2020 at 15:08, Aleksandra Mostowska (Miętus) < [email protected]> wrote:
it works perfectly well for short clips of this format, like 5-7 minutes. but bigger ones are problematic. let me know if you need help / tests :)
Best!
On Fri, 18 Dec 2020 at 14:36, רטו [email protected] wrote:
No, thank you, I don’t think this is needed. I’ll delve into the issue later today using a similar dummy file, which should be sufficient to reproduce and analyse.
BTW: QCTools 1.2 has been released a few hours ago.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748087869, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSOVQIATHBISZPXA3EDSVNLGTANCNFSM4UR6XLPA .
Hmmm
Stream #0:1: Data: none
Metadata:
file_package_umid:
0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28
data_type : audio
Data of data_type=audio. Do you know what these streams are supposed to be? Does VLC, mpv, etc play audio for this file?
You could adjust to ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -c copy h:\DNX33_145V013397006D_only_v.mxf which would make a video-only copy to test in qctools to rule out the video causing the issue. I am curious about the audio/data tracks though, it seems that ffmpeg doesn't identify it properly.
Hello, this command worked fine. Actually the new clip is a bit bigger: [image: image.png]
QCTools 1.2 get's stuck after a few seconds: [image: image.png] and does not proceede anymore. I left it like this and it shut down after some time. I noticed in taskmgr that it consumed all my available RAM (34Gb) so probably that is why it shut down.
Moreover I also spotted that it does not matter if I analyse i-frame (dnx) or (long g) xdcam codec. the issue is I think in op-atom pattern so some wrapping issues might be there. So the behaviour in both QCTools is the same. Are there any logs available that might help you?
Regards Aleksandra
On Fri, 18 Dec 2020 at 15:18, Dave Rice [email protected] wrote:
Hmmm
Stream #0:1: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28 data_type : audio
Data of data_type=audio. Do you know what these streams are supposed to be? Does VLC, mpv, etc play audio for this file?
You could adjust to ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -c copy h:\DNX33_145V013397006D_only_v.mxf which would make a video-only copy to test in qctools to rule out the video causing the issue. I am curious about the audio/data tracks though, it seems that ffmpeg doesn't identify it properly.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748107553, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSIECL4RLZMRN6ZRQBLSVNQCVANCNFSM4UR6XLPA .
Hello :) is there any update or can I help somehow with further reproduction?
Regards Aleksandra
On Fri, 18 Dec 2020 at 21:10, Aleksandra Mostowska (Miętus) < [email protected]> wrote:
Hello, this command worked fine. Actually the new clip is a bit bigger: [image: image.png]
QCTools 1.2 get's stuck after a few seconds: [image: image.png] and does not proceede anymore. I left it like this and it shut down after some time. I noticed in taskmgr that it consumed all my available RAM (34Gb) so probably that is why it shut down.
Moreover I also spotted that it does not matter if I analyse i-frame (dnx) or (long g) xdcam codec. the issue is I think in op-atom pattern so some wrapping issues might be there. So the behaviour in both QCTools is the same. Are there any logs available that might help you?
Regards Aleksandra
On Fri, 18 Dec 2020 at 15:18, Dave Rice [email protected] wrote:
Hmmm
Stream #0:1: Data: none Metadata: file_package_umid: 0x060A2B340101010101010F0013000000D0A79F9436E700A5A54DF4CEF3578F28 data_type : audio
Data of data_type=audio. Do you know what these streams are supposed to be? Does VLC, mpv, etc play audio for this file?
You could adjust to ffmpeg -i h:\DNX33_145V013397006D.mxf -map 0:v -c copy h:\DNX33_145V013397006D_only_v.mxf which would make a video-only copy to test in qctools to rule out the video causing the issue. I am curious about the audio/data tracks though, it seems that ffmpeg doesn't identify it properly.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bavc/qctools/issues/697#issuecomment-748107553, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASBMJSIECL4RLZMRN6ZRQBLSVNQCVANCNFSM4UR6XLPA .
Hi @amietus, no update yet and we're back in volunteer mode for development. @ElderOrb, have you had a chance to look at the memory issues listed?
I can reproduce on my end. (Sorry, I wasn’t on Windows for quite a time.)
Hi @amietus, very late reply, but qctools has had a significant refactor. Please test this issue against a recent daily build at https://mediaarea.net/download/snapshots/binary/qctools/. Will close this ticket for now, but please reopen if still an issue.