yi-hack-Allwinner icon indicating copy to clipboard operation
yi-hack-Allwinner copied to clipboard

Published 20 hours ago • verified publisher icon roleoroleo

RTSP with AAC?

Open andriej opened this issue 4 years ago • 23 comments

Hi,

Is there any way to enable/change PCM A-law (G711) codec of audio to AAC? G711 is not browser compatible and because of that HomeAssistant can't stream it further with audio.

I'd like to use one of cameras as baby monitor tho.

andriej avatar Jan 05 '21 17:01 andriej

At the moment you can choose ulaw, alaw and pcm. I don't know if live555 supports aac. I have to check it.

roleoroleo avatar Jan 05 '21 18:01 roleoroleo

All of those are 'linear' PCM codecs at least looking at Wikipedia while HA and browsers have to have AAC (source)

andriej avatar Jan 05 '21 18:01 andriej

The library doesn't support aac conversion. Maybe it should be implemented using this library: https://github.com/mstorsjo/fdk-aac

roleoroleo avatar Jan 08 '21 14:01 roleoroleo

Any chance to have it implemented in coming weeks? I can test if needed

andriej avatar Jan 08 '21 14:01 andriej

It's a long job. I have no time in this moment. Sorry.

roleoroleo avatar Jan 16 '21 17:01 roleoroleo

Ok, I understand. Could you just list points that could potentially be done, so maybe someone will want to dig further and get it implemented later on? Thanks

andriej avatar Jan 16 '21 21:01 andriej

Just to give it a try, I cross-compiled this: https://github.com/mstorsjo/fdk-aac and this: https://github.com/nu774/fdkaac

Then I tried to run fdkaac reading form the pcm fifo and it works. I can create an aac file. But the hardest part is getting it into the RTSP server.

roleoroleo avatar Jan 17 '21 12:01 roleoroleo

Could you just list points that could potentially be done, so maybe someone will want to dig further and get it implemented later on?

The fdk-aac library does most of the work. I think the simplest approach is to start from the ADTSAudioFileSource.cpp and ADTSAudioFileServerMediaSubsession.cpp files (from live555 library) and edit them to read the PCM data from the fifo and then convert it to AAC. The fifo is in /tmp/audio_fifo and it's in the following format:

  • 8 KHz
  • 16 bit
  • mono
  • signed
  • little endian

roleoroleo avatar Jan 17 '21 12:01 roleoroleo

Do you want to try this alfa? rRTSPServer.gz You can start the server with the following command: RRTSP_RES=both RRTSP_AUDIO=aac rRTSPServer

roleoroleo avatar Jan 26 '21 19:01 roleoroleo

it was a very complicated job. But funny.

roleoroleo avatar Jan 26 '21 19:01 roleoroleo

Sure!

I've ran it on one of cameras I have in garage for test. First thing I noticed was the speaker icon in the stream :-) Then after making few noises stream broke down with no visible error in console:

fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 167
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 172
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 173
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 260
fInargs.numInSamples 512
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 512
fOutargs.numInSamples 512
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 512
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0

I couldn't get server back up again after that, so I rebooted that camera. After SSH again - I did longer tests, voice seemed to be minimally out of sync from video (like before the image) but I would totally accept that and did not notice if not testing.

After like 2 mins of testing and the moment of changing IR (turned-off light), stream froze again and (I think) camera restarted as I got re-logged to SSH:

fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 186
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 216
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 187
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 1024
fOutargs.numInSamples 1024
outargs.numOutBytes 175
fInargs.numInSamples 0
fOutargs.numInSamples 1024
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
outargs.numOutBytes 0
fInargs.numInSamples 0
fOutargs.numInSamples 0
ErrorStatus 0

andriej avatar Jan 26 '21 20:01 andriej

One more thing - I didn't know where to put it (as ssh is quite limited on this device) so I cilled the other server running and just ran this one from /tmp. I'm not sure if it's not other watchdog killing or something.

Ok, just found it in /home.../bin/, so I've changed the file there and will continue test (but probably without AAC - or if I find way to pass variables there)

Edit2: Adding variables to /home/...etc/system.conf and changing binary make it not start even after reboot, so probably there's something more behind to be changed (unfortunately no logs I can find)

andriej avatar Jan 26 '21 20:01 andriej

You can overwrite the original rRTSPServer (make a backup copy). Then edit system.conf and set RTSP_AUDIO=aac

roleoroleo avatar Jan 26 '21 20:01 roleoroleo

Just did that and it works :-)

andriej avatar Jan 26 '21 20:01 andriej

I've moved the binary to the 2nd camera too - it worked, sound is there, but also I can hear like regular-distorted sound which may be somehow connected with transmission. Regular intervals, one by one, short. I'll try to record something tommrow.

Both of those cameras came together in one bundle (2in1)

andriej avatar Jan 27 '21 03:01 andriej

Try this binary: rRTSPServer.gz

roleoroleo avatar Jan 28 '21 20:01 roleoroleo

@roleoroleo here's the example recording from the camera making those sounds. I've updated the binary on both, working on both of them - only on one those sounds are present. Probably it may be also interference, will try to move the camera tommorow and see if it helps.

Besides that - both cameras work

andriej avatar Jan 29 '21 00:01 andriej

I've moved camera a little and the noise is much more silent, so it may be just interference.

andriej avatar Feb 04 '21 00:02 andriej

Fixed another bug: rRTSPServer.gz

roleoroleo avatar Feb 06 '21 17:02 roleoroleo

Works!

Is there any changelog tho? And will it be in main version any time soon? :-)

andriej avatar Feb 06 '21 19:02 andriej

I didn't commit it. But I will try to release with the next version.

roleoroleo avatar Feb 06 '21 22:02 roleoroleo

It's been a while - I guess it's within 0.2.5 already? Looking for some improvement regarding connectivity (debugging if it's wifi or camera itself right now).

andriej avatar Apr 15 '21 12:04 andriej

Yes, it's released within 0.2.5. About improvements see the release info.

roleoroleo avatar Apr 15 '21 13:04 roleoroleo

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Apr 09 '24 02:04 github-actions[bot]