Get frames without setting the format

[LairCortex]

Good afternoon! Our stream doesn't have sprop parameter attached so we can't pass processH264FmtpAttribute RtspMediaTrack check. Found a similar problem and solution in https://github.com/google/ExoPlayer/issues/10971#issuecomment-1431254450 , but the video failed without setting default parameters instead of checking processH264FmtpAttribute - formatBuilder.setWidth(1920).setHeight(1080); We can get parameters from the first frame, but we need to set default format, how can we get frames without setting the format

[microkatz]

Hi @LairCortex,

The RtspMediaTrack method still needs to provide a Format, just not with the sps parameters provided by the processH264FmtpAttribute method. You would then need to update the format information in the RtpH264Reader class with those values you get from that first frame. It is difficult to provide advice without seeing how you are customizing the code. Would you be able to provide an example repo showcasing the changes you are making?

[LairCortex]

Hi @microkatz, We have done the experiment and realise that we can catch the first packet in RtpExtractor.read it turns out we just need to update the Format Can you tell us how to do it correctly?

[microkatz]

@LairCortex

That's great! It would be in the processSingleNalUnitPacket(or other processing Nal unit method in RtpH264Reader). After receiving the sps data, you would need to call trackOutput.format(Format format) with a Format built from the sps data. Similar to the code below:

List<byte[]> initializationData = new ArrayList<>();
        initializationData.add(Arrays.copyOf(sps.nalData, sps.nalLength));
        initializationData.add(Arrays.copyOf(pps.nalData, pps.nalLength));
        NalUnitUtil.SpsData spsData = NalUnitUtil.parseSpsNalUnit(sps.nalData, 3, sps.nalLength);
        NalUnitUtil.PpsData ppsData = NalUnitUtil.parsePpsNalUnit(pps.nalData, 3, pps.nalLength);
        String codecs =
            CodecSpecificDataUtil.buildAvcCodecString(
                spsData.profileIdc,
                spsData.constraintsFlagsAndReservedZero2Bits,
                spsData.levelIdc);
        trackOutput.format(
            new Format.Builder(payloadFormat.format)
                .setSampleMimeType(MimeTypes.VIDEO_H264)
                .setCodecs(codecs)
                .setWidth(spsData.width)
                .setHeight(spsData.height)
                .setColorInfo(
                    new ColorInfo.Builder()
                        .setColorSpace(spsData.colorSpace)
                        .setColorRange(spsData.colorRange)
                        .setColorTransfer(spsData.colorTransfer)
                        .setLumaBitdepth(spsData.bitDepthLumaMinus8 + 8)
                        .setChromaBitdepth(spsData.bitDepthChromaMinus8 + 8)
                        .build())
                .setPixelWidthHeightRatio(spsData.pixelWidthHeightRatio)
                .setInitializationData(initializationData)
                .setMaxNumReorderSamples(spsData.maxNumReorderFrames)
                .build());

Do you have an sample RTSP server that we could use to test this as well?

[google-oss-bot]

Hey @LairCortex. We need more information to resolve this issue but there hasn't been an update in 14 weekdays. I'm marking the issue as stale and if there are no new updates in the next 7 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@LairCortex if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.