untrunc [aac @ 0x55bea6e39b60] channel element 3.6 is not allocated

[StuartJMackintosh]

Background

After recording a video file on an Android SD card and forgetting about the 4GB limit, my video file can't be played. The video file size is 4294967295.

Compiling

untrunc compiled on Debian 10 with: g++ -o untrunc -I./libav-12.3 file.cpp main.cpp track.cpp atom.cpp mp4.cpp -L./libav-12.3/libavformat -lavformat -L./libav-12.3/libavcodec -lavcodec -L./libav-12.3/libavresample -lavresample -L./libav-12.3/libavutil -lavutil -lpthread -lz -lvdpau -ldl -llzma -lbz2

Running

./untrunc  /tmp/VID_20191016_174120.mp4 /tmp/truncated.mp4 
Reading: /tmp/VID_20191016_174120.mp4
Opening: /tmp/VID_20191016_174120.mp4
Repair: /tmp/truncated.mp4
Track 0 codec: mp4a
mp4a: Success because of large s value.
[aac @ 0x5614bbbd2b60] channel element 3.6 is not allocated
Duration: 0

Invalid length: -1052488119. Wrong match in track: 0.
Track 1 codec: avc1
avc1: Failed for no particular reason.
Found 0 packets.

Saving to: /tmp/truncated.mp4_fixed.mp4
Track 0 (mp4a): duration: 0 timescale: 48000
Adjusting track duration to movie timescale: New duration: 0 timescale: 1000.
Track 1 (avc1): duration: 0 timescale: 90000
Adjusting track duration to movie timescale: New duration: 0 timescale: 1000.
Movie duration: 0 timescale: 1000

Related issues

This seems similar to #175 where the docker file was used instead of compiled untrunc and possibly #168

I would guess this is an issue with the audio track - are there alternative libraries to compile against?

Any suggestions are welcome!

[anthwlock]

I would guess this is an issue with the audio track - are there alternative libraries to compile against?

I once tried 'fdk-aac', but it requires known audio sample lengths. Haven't tried any other since.

Can you provide a small healthy file, which when used as both healthy and broken results in these type of errors?

[StuartJMackintosh]

Thanks for the quick response. I hadn't thought of using the healthy file for both <ok.mp4> and <corrupt.mp4>. Having now done this, I don't get any errors reported and it seems to have completed:

Track 0 (mp4a): duration: 338944 timescale: 48000
Adjusting track duration to movie timescale: New duration: 7062 timescale: 1000.
Track 1 (avc1): duration: 617993 timescale: 90000
Adjusting track duration to movie timescale: New duration: 6867 timescale: 1000.
Movie duration: 7062 timescale: 1000

So I created sample2.mp4 and it also works fine. Then truncated sample2, and was able to fix it with untrunc against the original, and against sample1.

Sample2 gives the same errors as sample1 if used as <ok.mp4> when trying to untrunc the original file.

Also have tried shortening the original file with dd if=truncated.mp4 of=sample-original-short.mp4 count=50000 and this gives the same result as originally reported which is Invalid length: -1052488119. Wrong match in track: 0.

This seems less likely to be related to aac as untrunc processes the samples just fine.

The samples were all created using the same Android phone as the original.

[StuartJMackintosh]

I should note that the failure happens immediately on invocation so that suggests that it is something in the header. When the file was shortened, the invalid length number was the same.

[anthwlock]

As I understand it:

• sample1 := orig_ok

fileOk	fileBad	result
orig_ok	orig_ok	ok
orig_ok	orig_bad	invalid length
orig_ok	sample2	ok
orig_ok	sample2_truncated	ok
sample2	orig_bad	invalid length
orig_ok_short	orig_bad	invalid length

If that's the case, it might be that orig_bad contains "garbage data" (at least partially). If so, the -s option of my fork might be helpful, give it a try.

Also look at orig_bad with a hex-editor: maybe the "garbage data" is visibly garbage (e.g. ascii encoded). In particular look at the problematic offset(s), where the error occurs.

[StuartJMackintosh]

Thanks @anthwlock

Files

I didn't help by switching file names - here is what I am working with:

File name .mp4	Size	description
truncated	4.0G	Original recording
sample1	19M	created on same device shortly after
sample2	34M	created on same device yesterday
sample2-trunc	20M	first 20Mb from dd
truncated-short	49M	first 50MB of truncated from dd

Tests

Tests and results with untrunc

<ok.mp4>	<corrupt.mp4>	Result
truncated	truncated	channel element 3.6 is not allocated / Invalid length
sample1	truncated	channel element 3.6 is not allocated / Invalid length
sample2	truncated	channel element 3.6 is not allocated / Invalid length
sample1	truncated-short	channel element 3.6 is not allocated / Invalid length
sample2	truncated-short	channel element 3.6 is not allocated / Invalid length
sample1	sample2	pass
sample1	sample2-trunc	pass
sample2	sample2-trunc	pass
truncated-short	truncated	Could not read at position

Will try the fork and look for garbage manually during today.

[StuartJMackintosh]

@anthwlock - your fork at https://github.com/anthwlock/untrunc with -s -sv and sample1 successfully recovered the video. This required additional package libavresample-dev and having a makefile made the build process easier.

The audio gradually strays from the video and is some way off at the end as you can see with the timings even with sv set.

Warning: guessed frame durations of 'avc1' will probably be wrong!
Info: Found 122649 packets ( mp4a: 80041 avc1: 42608 avc1-keyframes: 2780 )
Info: Duration of mp4a: 28min 27s 541ms  (1707541 ms)
Info: Duration of avc1: 28min 59s 718ms  (1739718 ms)
Warning: Unknown sequences: 33
Warning: Bytes NOT matched: 1.48MiB (0.03606%)
Info: saving truncated.mp4_fixed-s1-sv.mp4

Knowing this, can the audio be reduced to fit?

[ponchio]

In some videos the duration of the packets is not fixed but variable, and unfortunately this per packet duration is written in the header which is lost, hence the difference.

If the video (in this case) timing drift is more or less constant you can just slightly accelerate or slow the audio to match the video (it's way better to resample the audio if you don't care about exact timing)

It should actually be possible to sync the two streams (even if not exactly) using the fact that the camera writes audio and video packets interleaved in the file. I didn't think of it... and will add this feature.

[StuartJMackintosh]

Can the video be resampled using untrunc or will I need to separate, resample and combine? I tried -sv - stretches video to match audio duration (beta) which seemed to have no impact.

If you do have an update sometime soon whilst I still have this project open, I can test it for you and feed back.