Feature Request: MQTT - Audio detection through mqtt protocol.

[agsola]

Add audio detection in MQTT (as motion detection).

[roleoroleo]

Is it a feature available through the Yi app? If it's available then I could try to add to the firmware.

[agsola]

Only for baby crying AFAIK.

[roleoroleo]

I will check...

[PieVo]

Thinking about this too..

In TinyAlsa we can create a level-trigger but I did not dig deeper. Would it be possible to generate an IPC from within tinyalsa? This could then be picked up by the process handling them...

[roleoroleo]

Yes, it could be possible...

[roleoroleo]

What do you think about this piece of code?

    for (i = 0; i < pcm_bytes_to_frames(pcm, bytes); i++) {
        mono_buf[i] = *pcm_ptr & 0xffff;
        sum -= ((float) abs(sum_buf[i&511])) / 512.0;
        if (sum < 0.0) sum = 0.0;
        sum_buf[i&511] = mono_buf[i];
        sum += ((float) abs(sum_buf[i&511])) / 512.0;
        if (sum >= 10000.0) fprintf(stderr, "Sound detected!\n");
        pcm_ptr++;
    }

Within the same cycle of conversion to mono. With:

    int16_t sum_buf[512];
    float sum;

    memset(sum_buf, '\0', sizeof(sum_buf));

512 samples @ 16 KHz ---> 32 ms of evaluation period. Threshold 10000

[PieVo]

What is this part for?

"sum -= ((float) abs(sum_buf[i&511])) / 512.0;"

Personally, I would avoid floats in the non-float tinyalsa lib and evaluate a single block of data. What I've seen is that the block size is always 512 frames...

config values: sensitivity = 100 threshold = 50

for loop i over samples { mono_buf[i] = pcm_ptr & 0xffff; / Mono conversion */

if (abs(mono_buf[i]) > sensitivity) if (--threshold <= 0) notify_audio_detected(); }

[roleoroleo]

What is this part for?

"sum -= ((float) abs(sum_buf[i&511])) / 512.0;"

sum -= and sum += are to calculate a "sliding average" and compare it with the threshold (not every sample). I think it's not so common to have 50 successive samples over the threshold, probably it is better to do an integral over a period. And also to detect a sound across two blocks.

No problem to avoid floats.

[PieVo]

Ah like that... Kind of the energy-level per block. Multiple block might be useful indeed and you ignore the variable amount of background noise per location 👌

I think this feature would be more generic/useful than just the baby-cry detection.

On Mon, May 11, 2020 at 1:00 PM roleo [email protected] wrote:

What is this part for?

"sum -= ((float) abs(sum_buf[i&511])) / 512.0;"

sum -= and sum += are to calculate a "sliding average" and compare it with the threshold (not every sample). I think it's not so common to have 50 successive samples over the threshold, probably it is better to do an integral over a period. And also to detect a sound across two blocks.

No problem to avoid floats.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/roleoroleo/yi-hack-MStar/issues/120#issuecomment-626632803, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNA2O5J2A7L7WBHBZLXXGDRQ7LFLANCNFSM4LRHGCBQ .

[roleoroleo]

I'm going to do some experiments tonight...

[roleoroleo]

Yes, it works. I could add it in the next release.

[jd1900]

Yes, it works. I could add it in the next release.

Independently of turning the camera on in the app? That would be great news! Now, unless you start the camera, the movement detection doesn't work.

[roleoroleo]

Independently of turning the camera on in the app?

I don't know.

[Xandrios]

Yes, it works. I could add it in the next release.

That sounds awesome. Any chance of porting to the Allwinner branch as well?

[roleoroleo]

Yes, it's very similar.

[thoemmes86]

Is there any update to this enhancement? It would be really helpful to use the dome x as baby monitor. At the moment, the "Abnormal Sound" event is not transferred via MQTT.

[roleoroleo]

I completely forgot about this. I will try to add this feature and I will send a beta to test.

[thoemmes86]

I completely forgot about this. I will try to add this feature and I will send a beta to test.

That would be perfect. Happy to test it. While in earlier YI Settings there was the BabyCry detection enabled, the current version offer the setting to enable abnormal noise detection and a detection sensitivity level in dB.

[roleoroleo]

What's your model suffix?

[thoemmes86]

What's your model suffix?

YI Dome Camera X (y30)

A site remark if helpful. I bought a Yi 1080p Home QFUS which I flashed with Allwinner2 today. There the feature is already working as expected.

[roleoroleo]

Here it is: y30_0.5.2.tar.gz

[thoemmes86]

Here it is: y30_0.5.2.tar.gz

I had some struggles to install the firmware. It lead to having a situation that the webinterface was not working excl. the start page shows only the top navigation but nothing else. After reverting to some 4.x version und tray it again, it seams to work.

Now while your version is installed, the settings 'Sound Detection' and 'Sound detection sensitivity' are not saved while clicking on save. Other settings are working fine. So it is not possible to activate the functionality in the web interface.

[roleoroleo]

Did you try to clean the cache of the browser? I changed javascript files. This could explain both problems.

[thoemmes86]

Now while your version is installed, the settings 'Sound Detection' and 'Sound detection sensitivity' are not saved while clicking on save. Other settings are working fine. So it is not possible to activate the functionality in the web interface. --> Browser Cache solves this issue.

Never the less, I have issues installing the hack via SD card. Tested this on a second Dome X cam and faced the same issue installing the adapted 0.5.2. Workaround was installing 0.4.2 and than 0.5.2, works fine while all settings are gone. May there is a glitch.

Sound detection seems to working fine. What I recognized is, that while the camera is turned off in the settings, the sound detection still is active. Same behaviour with the latest Allwinner-v2 version 0.2.9.

[roleoroleo]

What I recognized is, that while the camera is turned off in the settings, the sound detection still is active.

This thing is actually not managed.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.