Request for example of playing a midi file

[owenlamont]

I wasn't sure if playing midi files is something ipytone even supports, but if it is, I'd love an example of loading and playing a midi file.

[benbovy]

There's no built-in support (yet) in ipytone for playing midi files, but you could parse the midi file using, e.g., mido to create ipytone.Part events without much effort. For example:

import ipytone
import mido

# convert MIDI note numbers to notes (string notation)
# modified from https://gist.github.com/devxpy/063968e0a2ef9b6db0bd6af8079dad2a

NOTES = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']
OCTAVES = list(range(-2, 9))
NOTES_IN_OCTAVE = len(NOTES)

def number_to_note(number):
    octave = number // NOTES_IN_OCTAVE - 2
    assert octave in OCTAVES, errors['notes']
    assert 0 <= number <= 127, errors['notes']
    note = NOTES[number % NOTES_IN_OCTAVE]

    return note + str(octave)

# read midi file and extract 1st track
mid = mido.MidiFile("filename.mid")
track = mid.tracks[0]

# use 120 BPM by default (possible to read it from the midi file)
tempo = 500_000
ipytone.transport.bpm.value = mido.tempo2bpm(tempo)

# parse midi note messages and create ipytone.Part events
current_time = 0.
current_notes = {}
events = []

for msg in track:
    current_time += mido.tick2second(msg.time, mid.ticks_per_beat, tempo)
    
    if msg.type == "note_on":
        current_notes[msg.note] = {
            "time": current_time,
            "note": number_to_note(msg.note),
            "velocity": msg.velocity / 127,
        }
    elif msg.type == "note_off":
        event = current_notes.pop(msg.note)
        event["duration"] = current_time - event["time"]
        events.append(event)

Then create the partition and play it (e.g., with an ipytone.PolySynth):

msynth = ipytone.PolySynth(volume=-8).to_destination()

def clb(time, note):
    msynth.trigger_attack_release(
        note.note, note.duration, time=time, velocity=note.velocity
    )

part = ipytone.Part(callback=clb, events=events)

part.start()
ipytone.transport.start()

Not sure that we should add mido as a dependency here, but such example would be a nice addition in the docs.

[benbovy]

In the mid/long term, it would be nice to have features like GUI widgets to visualize (#12) and/or edit partitions as well as widgets to connect MIDI devices using https://webmidijs.org/ (a bit like https://github.com/jupyter-widgets/midicontrols but generalized to any device), probably in a 3rd party package.

[owenlamont]

Thanks for the example! That alone is great, I was just curious how to make it work, with or without the support of another package.

[owenlamont]

I'll have to play with it more but the example appears to be working well, only tweak I had to make was to the:

current_notes.pop(msg.note)

line. The midi I tested on apparently had note off without a corresponding note on which threw a KeyError which I just caught and skipped the event with.

[benbovy]

I don't have much experience with midi files, maybe this could happen when starting to record a performance while a key is already pressed down? I guess a "note on" without a "note off" could happen as well, then?

Probably a better approach would be to use the instrument trigger_attack() and trigger_release() methods in the callback instead of computing the note duration and use trigger_attack_release(). This is not possible right now, though, but it can be easily fixed:

• ipytone.PolySynth.trigger_release() is missing a notes argument (required in Tone.js)
• ipytone.Note should have an attribute used to call either trigger_attack or trigger_release in the callback (in Tone.js the structure of that callback argument is arbitrary and thus much more flexible).

[benbovy]

With #104 and #105 (available in the next release), creating a Part object from a midi file is slightly more straightforward than in https://github.com/benbovy/ipytone/issues/77#issuecomment-1161666380:

events = []

for msg in track:
    current_time += mido.tick2second(msg.time, mid.ticks_per_beat, tempo)

    if msg.type not in ["note_on", "note_off"]:
        continue

    event = {
        "time": current_time,
        "note": number_to_note(msg.note),
        "velocity": msg.velocity / 127,
        "trigger_type": "attack" if msg.type == "note_on" else "release"
    }
    events.append(event)

msynth = ipytone.PolySynth(volume=-8).to_destination()

def clb(time, note):
    msynth.trigger_note(note, time)

part = ipytone.Part(callback=clb, events=events)

I think this solution should work without needing any additional check. One drawback is when ipytone.transport is stopped between the scheduled attack and release of some note, that note is never released (silencing the instrument would require a manual call to trigger_release() or release_all() for polyphonic instruments).