Is it possible to force h264 in non SIP calls?

[zevarito]

pipecat version

0.0.84

Python version

3.11

Operating System

Ubuntu

Question

I am looking for a way to ensure video encoding to be forced to h264, thank you.

What I've tried

No response

Context

No response

[markbackman]

Which transport are you using? Daily?

[zevarito]

Which transport are you using? Daily?

sorry I forgot to mention! I am using Daily

[markbackman]

Daily exposes H264 through daily-js, but it's advanced and generally advise to use VP8 for all things video as it's widely adopted/compatible and very stable. Nonetheless, here's where you can find it in the types: https://github.com/daily-co/daily-js/blob/daily-js-releases/index.d.ts#L403-L426

This can be specified as dailyConfig when creating your CallClient: https://docs.daily.co/reference/daily-js/daily-call-client/properties#properties

Are you using daily-js or one of Pipecat's SDKs?

[zevarito]

In my experience h264 is better codec than VP8 and part of the WebRTC standard as well, I would prefer to use it for my case, but also would be happy to use AV1 and fallback to VP8, if you guys can expose a way to prioritize codecs would be great!

I believe the issue arises if a non daily-js peer joins the room first. Could be the sfu tweaked with room params?

[markbackman]

I believe the issue arises if a non daily-js peer joins the room first.

Which client SDK(s) are you using?

[zevarito]

Which client SDK(s) are you using?

Python SDK with Pipecat.

by "non daily-js..." I meant that there is not possible to prioritize h264 with Python SDK it seems.

[markbackman]

Maybe @aconchillo or @filipi87 recalls what is support by daily-python.

[filipi87]

It should be possible to define the codec in the publishing settings. Here are the docs:

• https://reference-python.daily.co/types.html#videopublishingsettings

You should be able to invoke this using Pipecat’s update_publishing method.

[zevarito]

You should be able to invoke this using Pipecat’s update_publishing method.

Thanks, I am gonna take a look!

[zevarito]

Hi @filipi87 I've tested this without success. It seems that you are allowed to call update_publishing with the pipeline already running, that's why I've tried to set the preferrer codec on on_first_participant_joined and also in StartFrame with a custom processor, none of those approaches worked for me.

What I am doing:

await transport.update_publishing({
    "camera": {
        "sendSettings": {
            "preferredCodec": "H264"
        }
    }
})

There is a most precise way to force the codec instead of suggest to avoid what the SFU or another peer might prefer? Is it possible to push encoded video frames instead of raw ones?

Thanks for you help!