segmentation fault in read_slice_layer_rbsp

[Serjyk]

Hello. I caught a segmentation fault while working with an H264 video. This happens after calling the methods find_nal_unit(...) and read_nal_unit(...) with the next set of bytes in hex: "00000001419a246c437ffea7840000030000097800000001"

Application crashes in class h264_stream.c of method read_slice_layer_rbsp(...). Variable slice_data->rbsp_size is equal to negative value when we perform memcpy(...).

[MitchellHansen]

I had a similar problem, try using the read_debug_nal_unit and see if it still crashes. There appears to be some desync between the debug/non-debug code.

If that fixes it for you, modifying read_nal_unit to match read_debug_nal_unit was all that was needed to fix it (for me)

        if ( slice_data->rbsp_size > 0 )
        {
            slice_data->rbsp_buf = (uint8_t*)malloc(slice_data->rbsp_size);
            memcpy( slice_data->rbsp_buf, sptr, slice_data->rbsp_size );
            // ugly hack: since next NALU starts at byte border, we are going to be padded by trailing_bits;
            return;
        }
        else
        {
            slice_data->rbsp_buf = NULL;
            slice_data->rbsp_size = 0;
        }