No audio in output file when audio stream at index 1

[UPuXA]

When i process a mp4 video with the audio stream at index 1 the audio is not in the final file.

I use the feature/package-installation branch and cli.py with the following parameters:

weights="1080p_medium_v8",
blur_workers=5,
blur_size=9,
threshold=0.2,
roi_multi=2,
quality=5,
feather_edges=10,
no_faces=False,
blur_memory=1,
batch_size=8,
export_mask=False,
export_colored_mask=False,
export_json=False

Tests

Test method: ffprobe -show_streams -v quiet -show_format -print_format json PATH

Results

Summary

Format: index codec_type codec_name

Ok audio:

original file 0 video h264 1 audio aac

processed file 0 video h264 1 audio aac

Missing audio:

original file 0 audio aac 1 video h264

processed file 0 video h264 1 video h264

Detailed results

Excerpts of ffprobe

Ok audio:

"index": 0, "codec_name": "h264", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "profile": "High", "codec_type": "video", "codec_tag_string": "avc1",

"index": 1, "codec_name": "aac", "codec_long_name": "AAC (Advanced Audio Coding)", "profile": "LC", "codec_type": "audio", "codec_tag_string": "mp4a",

after process

"index": 0, "codec_name": "h264", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "profile": "High", "codec_type": "video", "codec_tag_string": "avc1",

"index": 1, "codec_name": "aac", "codec_long_name": "AAC (Advanced Audio Coding)", "profile": "LC", "codec_type": "audio", "codec_tag_string": "mp4a",

Missing audio:

"index": 0, "codec_name": "aac", "codec_long_name": "AAC (Advanced Audio Coding)", "profile": "LC", "codec_type": "audio", "codec_tag_string": "mp4a",

"index": 1, "codec_name": "h264", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "profile": "High", "codec_type": "video", "codec_tag_string": "avc1",

after process

"index": 0, "codec_name": "h264", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "profile": "High", "codec_type": "video", "codec_tag_string": "avc1",

"index": 1, "codec_name": "h264", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "profile": "High", "codec_type": "video", "codec_tag_string": "avc1",

[UPuXA]

Hello, i searched a little bit myself and found this stack overflow answer. The ffmpeg command allows not only absolute stream specifiers like "stream 0", but also "audio stream 0". In the command it would be written like so: "0:0" vs "0:a:0".

The current version uses the absolute specification https://github.com/tfaehse/DashcamCleaner/blob/6a9160b85aa294f35f540665f53420267863e2b1/dashcamcleaner/src/blurrer.py#L113-L131

This could be the fixed version

 subprocess.run(
    [
        ffmpeg_exe,
        "-y",
        "-i",
        temp_output,
        "-i",
        input_path,
        "-c",
        "copy",
        "-map",
        "0:v:0",
        "-map",
        "1:a:0",
        "-shortest",
        output_path,
    ],
    stdout=subprocess.DEVNULL,
)

(ps.: great work und sorry, i have no idea how to make a pull request)

[joshinils]

@UPuXA you can edit files in browser and propose changes like that, I have done so just yet in the above-mentioned commit.

Please test the change if it works correctly, @UPuXA (or @tfaehse).

[UPuXA]

Hello again, i tested it and it works, i have now audio, the file size is significantly smaller and i want to share some additional findings.

It seems like the original/current behavior copies the complete original video stream into the output file if the video stream is at index 1 in the original file. With the current version i was able to delete stream 0 (the processed one) to see the source file (without the blurring). I verified this by disabling this whole segment by changing the if audio_present statement to if False: https://github.com/tfaehse/DashcamCleaner/blob/6a9160b85aa294f35f540665f53420267863e2b1/dashcamcleaner/src/blurrer.py#L112-L140 The file was smaller and the original stream gone.

Like i wrote the file size is now significantly smaller too (18,3MiB vs 248,0Mib) because of the missing source stream. It seems like there is some heavy compression going on in the inside and no parameter to change this "behavior" (i dont really care and maybe my original action cam video files are garbage).

However Big thanks for your help @joshinils

---

Correction:

It seems like there is some heavy compression going on in the inside and no parameter to change this "behavior"

--quality [1.0, 10.0]
                Quality of the resulting video. higher = better. Conversion to crf: ⌊(1-q/10)*51⌋.