docs: Add Sound Effects documentation.

[microbit-carlos]

Proposal

The current proposal adds a new class in the audio module named SoundEffect, which can be played using the audio.play() function.

The user can then create new instances, configuring them via constructor arguments, and after creation they can be modified via attributes.

A collection of a Sound Effects could be played by grouping them in a list (or something more advanced like a generator, so essentially an iterable).

Preview build of the docs: https://microbit-micropython--753.org.readthedocs.build/en/753/audio.html

In progress

• [x] The default values have not been considered yet, the current values are more or less random
  • More info in this comment: https://github.com/bbcmicrobit/micropython/pull/753#issuecomment-1172629876
• [ ] The descriptions and explanations could use some work, as the document is getting long a bit hard to digest, but we can update that later
• [ ] The order of the constructor arguments could be modified if it makes more sense to group them differently
• [ ] The built-in effects (if they are not dropped) are not defined yet. MakeCode has these examples, which we could reuse or adapt/modify
  • https://github.com/microsoft/pxt/blob/v8.1.1/webapp/src/components/soundEffectEditor/soundUtil.ts#L52-L182
  • We should reduce the total number of provided Sound Effects, as their usefulness is limited (compared with their availability in the editors) and take additional flash space
  • We are currently planning to play with this feature at europython and attendees can then submit their Sounds Effects

Things Still Under Consideration

1. Effect() might not be the best name for this feature
   • We might want to use the word "effect" for other future features, like filters than could be applied to recorded sound before it is played back
   • SoundEffect() is better but also longer (and we prefer shorter names that are easier to type for young learners)
   • This PR description has been updated already to reflect the SoundEffect name as described in comment https://github.com/bbcmicrobit/micropython/pull/753#issuecomment-1172629876
2. The parameter name interpolation is too long, we are still looking for alternatives.
   • curve is being considered, but one of the options is actually INTER_CURVE, so we might have to rename that to something else
   • ramp?
3. The idea to provide built-in pre-created Sound Effects was to let the user easily create and modify an Effect. In the MakeCode Editor and Python Editor the user can select one from a list, and the code/block is automatically generated showing all the values used to create that sound.

Alternative Proposals

A new function to play an effect instead of creating a class instance

We know that some beginner programmers struggle with the concept of instantiating classes in variables, and then using attributes to modify them. An example of this would be NeoPixels, although on the other hand things like Images where a new instance is immediately used in a function has generally been okay. So:

# This has generally been more difficult
np = neopixel.NeoPixel(pin0, 8)
np[0] = (255, 0, 128) 
np.show()

# But this has been okay, which is similar to audio.play(Effect(...))
display.show(Image('00300:'
                   '03630:'
                   '36963:'
                   '03630:'
                   '00300'))

So we are considering providing a function to play a single Sound Effect:

# Configure the effect on the accelerometer data
my_effect = audio.play_effect(
    freq_start=400,
    freq_end=2000,
    duration=500,
    vol_start=100,
    vol_end=255,
    wave=audio.WAVE_TRIANGLE,
    fx=audio.FX_VIBRATO,
    interpolation=audio.INTER_LOG
)

# And using sensor data
while True:
   audio.play_effect(
        freq_start=accelerometer.get_x(),
        freq_end=accelerometer.get_y(),
        duration=500,
        vol_start=100,
        vol_end=255,
        wave=audio.WAVE_TRIANGLE,
        fx=audio.FX_VIBRATO,
        interpolation=audio.INTER_LOG
    )

If we adopt this we might need a better name that play_effect(...) as it is not that different than play(Effect(...)).

Implementation details not shown in the docs

• Classes normally have a longer "internal" name (e.g. Image-> MicroBitImage, so the Effect class could have the longer MicroBitSoundEffect internal name
• Built-in pre-made Effects should be immutable
• Serialisation of a Sound Effect, for example to be sent via radio, could be done in a string format that could be eval'ed directly
  • The provided constants depend on the way the audio module is imported (import audio vs from audio import * vs from microbit import *), so their value would have to be used directly
    • For these values to be human readable we might need to use strings
  • i.e. print(audio.Effect(vol_start=100) -> Effect(freq_start=400, freq_end=2000, duration=500, vol_start=100, vol_end=255, wave="triangle", fx="vibrato", interpolation="log")
  • If the worse case scenario doesn't fit in the default radio packet size (or the max size), maybe we could use positional arguments for __repr__ and keyword arguments for __str__?
    • The example from before would become Effect(None, 400, 2000, 500, 100, 255, "triangle", "vibrato", "log")
• Not all the CODAL effect parameters are exposed to the user, for example, the fx has parameters that can be tweaked.
  • The hardcoded parameters can be found in the MakeCode implementation: https://github.com/microsoft/pxt-microbit/blob/v5.1.1/libs/core/soundexpressions.ts#L370
• By default audio.play() sets wait=False so it's a blocking function. If the CODAL call async version of play is called, and multiple calls are done inmediately one after another, they will not overlap.
  • So these three sound effects will still play one after another

    audio.play(audio.Effect.CHIRP, wait=False)
    audio.play(audio.Effect.CHIRP, wait=False)
    audio.play(audio.Effect.CHIRP, wait=False)
    

    But this will show the image while playing the sound

    audio.play(audio.Effect.CHIRP, wait=False)
    display.show(Image.HAPPY
    

  • We'll need to figure out how to document it, or if there is anything that can be done about this.

[microbit-carlos]

@dpgeorge @jaustin @microbit-giles This PR is ready for review.

As we'd like not to delay the implementation too much it'd be appreciated if any feedback could be provided soonish.

[microbit-carlos]

Oh, and @microbit-matt-hillsdon, I think you might be interested to have a look at these as well at some point for the stubs.

[microbit-giles]

My main concern is that the name Effect() may cause confusion and problems later on, especially if we implement sound 'filters' like reverb, amplification and reverse which are known as 'Effects' in widely-used sound editing software. We also have an 'fx' parameter within Effect(). Although longer to type, I think SoundEffect() may be clearer and less ambiguous in future.

[microbit-matt-hillsdon]

Oh, and @microbit-matt-hillsdon, I think you might be interested to have a look at these as well at some point for the stubs.

Thanks, yes, I think I'd like to have a stubs branch for this so we can spot any issues typing the API early.

[microbit-carlos]

Okay, based on @microbit-giles and @jaustin suggestion I've renamed Effect() to SoundEffect() in b89aa5d0f53cba0147e665da880c945f229b9246

Also updated the default values in ce19f55b4a16f2507e3d3e277a77dcfccf988892 to produce this sound: https://makecode.microbit.org/_ajAW04hDL0MP As these are the default values, that will be the sound played with audio.play(audio.SoundEffect())

[microbit-carlos]

@dpgeorge I think we are at a point were we can implement the current version of the docs (minus the addition of the pre-made built-in SoundEffects), so that we can start playing with the feature and get a feeling for it.

For that I think it might be good to keep the implementation in a feature branch, so that we can still tweak it before merging it.

Also we would like to of bringing the Python Editor beta with a MicroPython build with Sound Effects for workshops at EuroPython. I think it would be really cool if we could play with the SoundEffects with people at the conference and if they'd like to contribute their sounds to an example file, we could then pick some of those to use as the pre-build SoundEffects and credit them. The conference starts on the 11th of July, do you think that is doable? Ideally with a bit of time for us to be able to make sure any workshop material we prepare works.

[dpgeorge]

The parameter name interpolation is too long, we are still looking for alternatives

How about shape? That matches the internal CODAL name.

[dpgeorge]

The existing Sound class lives in microbit.Sound, and its microbit.audio.AudioFrame. So where should SoundEffect go, in microbit or microbit.audio?

[microbit-carlos]

The existing Sound class lives in microbit.Sound, and its microbit.audio.AudioFrame. So where should SoundEffect go, in microbit or microbit.audio?

The original proposal was to have these inside the audio module, as it's related to things like the audio.AufioFrame. So it ends up looking like this: audio.play(audio.SoundEffect())

The reason we ended up putting Sound in the microbit module, instead of the audio module was:

1. We considered these sounds to be micro:bit specific, to be part of the micro:bit personality, i.e. playing the "micro:bit HAPPY sound".
2. Shorter (audio.Sound.HAPPY vs Sound.HAPPY) always helps

For now the test branch has it as audio.SoundEffect, but we should reconsider if there is a good use case for having it as microbit.SoundEffect.

[microbit-carlos]

Merging this as it is the current state of SoundEffect in the v2.1.0-beta.1 release. Will create a new PR to include proposals on predefined sounds.

[microbit-carlos]

• https://github.com/bbcmicrobit/micropython/pull/765