Mean absolute value of [-1,+1] samples for visualizers

[jsalsman]

The visualizer vaveform averages of float samples in [-1, 1] look terrible, because their absolute value was not used in computing the mean. ~~You might also consider sending the cube root of these means, because the typical value is so small. I.e.,~~ [EDIT: there's a much better adjustment in my comment below.]

gain: this.visualSign * Math.pow(this.sampleSum / this.framesSinceLastPublish, 0.333),

this.visualSign is used to show each agregate with alternating signs so the visualization looks like an actual sound pressure wave instead of just the top half of one.

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[jsalsman]

you'll need to sign a Contributor License Agreement (CLA).

Done as indicated in the updated check at https://github.com/GoogleChromeLabs/web-audio-samples/pull/350/checks?check_run_id=18952386342

[jsalsman]

This works great for remapping the mean absolute samples. Don't ask me how, ChatGPT came up with it.

          } else if (event.data.message === 'UPDATE_VISUALIZERS') {
            let expKx = Math.exp(15 * event.data.gain);
            document.getElementById('level').value = 
              ((expKx - 1) / (expKx + Math.E)).toFixed(2);
          }

You can experiment with different values of 15 until you get the range you like.

[mjwilson-google]

Thanks for working on this! I am planning to do a full review, but for an initial comment: the types of waveform visualization I've usually seen are either envelope (which would be max/min over a range) or RMS (which is always non-negative, perhaps what this is going for). I'll look into how this is being used and may have some suggestions.

[mjwilson-google]

I finally got a chance to test this, and it looks like the sampleSum value is actually not used anywhere. It is passed in the UPDATE_VISUALIZERS message as a gain parameter, but that parameter is never read in app.js or elsewhere. So, the behavior of the visualizer is actually the same before and after this change.

I agree that the waveform display should be updated to something closer to standard audio software. We should also remove unused values such as sampleSum from the codebase since they're confusing. I will file issues to do this.

I would like to close this PR, but please feel free to comment or reopen if I missed something. And thank you again for taking the time to write a patch and bring this issue up!

[jsalsman]

it looks like the sampleSum value is actually not used anywhere. It is passed in the UPDATE_VISUALIZERS message as a gain parameter, but that parameter is never read in app.js or elsewhere. So, the behavior of the visualizer is actually the same before and after this change.

Did you try it with and without? It definetly ends up in https://github.com/GoogleChromeLabs/web-audio-samples/blob/main/src/library/Waveform.js#L61 and https://github.com/GoogleChromeLabs/web-audio-samples/blob/main/src/library/VUMeter.js#L67 as this.dataArray_ but I don't understand modern javascript well enough to understand how.

I'll reopen on the theory that you didn't test it, and will probably want the change if you do.

[jsalsman]

On second thought, it's not clear to me that I was depending on the code in https://github.com/GoogleChromeLabs/web-audio-samples/pull/350#issuecomment-1824828745 which I didn't have in the PR fileset, so I will leave closed and trust that you will overhaul things.

@mjwilson-google by the way, I used this code to make a MUCH nicer recorder with a far better VU meter, and voice activity detection, and more; plus I open sourced it under the MIT License, so you can use as much or as little as you like here. Please try it: https://webrec.replit.app/record

[mjwilson-google]

Thanks for the link to your application! I gave it a try and it seemed to work well.

I have written some audio visualizers in the past, so I think I already have an idea of how to fix this up. Of course, you are welcome to continue contributing code here if you would like.

Regarding other comments:

I did test the example with and without the patch. I also confirmed by commenting out the setting of the gain parameter: this should result in undefined symbol messages logged to the console if it's used anywhere but I didn't observe any (I also added a new usage of gain and did observe these messages, just to check).

It's possible that when this PR was raised the gain parameter was used, but it isn't anymore.

The dataArray_ members are used to store the output of the AnalyserNode (see https://webaudio.github.io/web-audio-api/#dom-analysernode-getfloattimedomaindata). The analyser is connected directly to the microphone signal here: https://github.com/GoogleChromeLabs/web-audio-samples/blob/main/src/audio-worklet/migration/worklet-recorder/app.js#L91