Multiple MIDI virtual input ports

[cyberic99]

Hi,

I use python-rtmidi with several MIDI input ports at the same time.

So I create multiple rtmidi.MidiIn(), open a port, and set a different callback for each midiIn.

Is it the right way to do it?

because when I some messages simultaneously on several ports, I frequently get: MidiInAlsa::alsaMidiHandler: unknown MIDI input error!

Using strace, I notices that reading from ALSA ports returns EAGAIN on these cases.

I tried to change the input queue size, and to put locks inside the callback function, but it does not help.

Do you know if rtMidi supports this?

Thanks

[cyberic99]

also: setting an error callback seems ignored in this case. I just get a messages printed to stderr

[SpotlightKid]

Code please.

[cyberic99]

I'll try to provide a simple reproducer asap

[cyberic99]

I can reproduce it after 1 or 2 minutes, by simultaneously and repeatedly flooding the different ports with these midi bytes:

'0x90 0x3c 0x40'

My original code is more complex, and this happens more frequently

I get this message:

MidiInAlsa::alsaMidiHandler: unknown MIDI input error!
System reports: Resource temporarily unavailable

#!/usr/bin/env python3

import time
import rtmidi
import queue

class Context:
    def handle_midi_in(self, event, data):
        message, deltatime = event
        q.put(event)
        #print("p1", event)

    def __init__(self, pn, q):
        self.pn=pn
        self.q = q
        self.mi = rtmidi.MidiIn(name=pn)
        self.mi.ignore_types(sysex=False, timing=True, active_sense=True)
        self.mi.open_virtual_port(name=pn)
        self.mo=rtmidi.MidiOut(name=pn)
        self.mo.open_virtual_port()
        self.mi.set_client_name(pn)
        self.mo.set_client_name(pn)
        self.mi.set_callback(self.handle_midi_in)

q = queue.Queue()

c1 = Context("p1", q)
c2 = Context("p2", q)
c3 = Context("p3", q)
c4 = Context("p4", q)



while True:
    m = q.get(1)
    #print(m)

I am using python 3.9.1 on archlinux

[SpotlightKid]

Thanks for the test code. At first glance it looks correct.

I'll look into it as soon as I can (some time this week).

[cyberic99]

thanks. I don't know what it my code makes it happen more frequently.

hopefully we'll find out.

I hope it is not a bug somewhere in rtmidi.

But at least, the fact the the error callback is not called looks suspicious...

[cyberic99]

hi @SpotlightKid, did you manage to reproduce this issue?

[SpotlightKid]

@cyberic99 Sorry, no, I didn't have time yet. I currently have to look for paid work.

[SpotlightKid]

I finally had time to try out your script. Unfortunately, I could no reproduce the errors you have.

(I'm on Manjaro, using Python 3.9.1 too).

How much load are we talking about here?

I created 4 MIDI players (mamba) and loaded a heavy MIDI file in each and then routed it's output into the virtual input ports of your script and played all four together. Even if I let it run for 5+ minutes, I don't see a single error message.

[cyberic99]

Hello @SpotlightKid . and thank you for taking some time to try to repoduce my issue.

The software I am using is using MIDI as a transmission protocol, so we send a lot of MIDI data in short bursts. It can go up to thousands of MIDI messages per minute.

To reproduce the issue I used this program, which uses mido:

miditool.py.zip

Like this:

./miditool.py  -p "p1:p1" --hex '0x90 0x60 0x00' --flood

And I quickly get:

MidiInAlsa::alsaMidiHandler: MIDI input buffer overrun!

and also:

MidiInAlsa::alsaMidiHandler: unknown MIDI input error!
System reports: Resource temporarily unavailable

[SpotlightKid]

So I managed to reproduce the error messages you were getting using your miditool.py script.

I'm not sure how to assess this issue, though.

With your script, on my system, I can send ~100K MIDI messages per second. Apparently, Python is just not fast enough to keep up with this and messages arrive faster in the ALSA input buffer than RtMidi is able to retrieve them. I guess the Python callback function just takes too long. I briefly tested the script also with the cmidiin program from the tests directory of the rtmidi sources as the receiver, and I didn't get error messages then. Though I'm not sure whether there really were no errors or they just did not get printed.

But, honestly, this speed is out the range that MIDI was ever intended for. If I insert even a very short (0.00001s) sleep in the while loop in the script and thus bring down the message rate to ~10K/seconds, I get no errors on the receiving side anymore.

I'm thus tempted to close this as "wontfix" and just advise you to put in some rate-limiting in your sending code.

[cyberic99]

Hello and thank you for taking the time to try again.

did you get the 'input buffer overrun' or the 'Resource temporarily unavailable' ?

As I said, my software is not sending at 100Kmsg/s for long, it only sends data in small bursts.

But it seems that having several virtual MIDI IN ports in the same process makes the message appear more often.

And on my side, it is hard to do some rate limiting, as:

• I am trying to keep the latency low
• sending MIDI is done by multiple processes

But yeah, I understand this is not a 'normal' use case and also that MIDI is usually limited to 33000b/s.

[SpotlightKid]

did you get the 'input buffer overrun' or the 'Resource temporarily unavailable' ?

Both.

my software is not sending at 100Kmsg/s for long, it only sends data in small bursts. I am trying to keep the latency low

Like I mentioned, I had no problems with ~10K msg/sec. Even if you keep the rate of incoming messages well below that, the latency shouldn't even reach the millisecond range.

[SpotlightKid]

also: setting an error callback seems ignored in this case. I just get a messages printed to stderr

That's a bug in RtMidi. In the alsaMidiHandler function it uses std::cerr << "..." instead of the proper error function.

[cyberic99]

did you get the 'input buffer overrun' or the 'Resource temporarily unavailable' ?

Both.

so you think the 'Resource temporarily unavailable' message shows up because I send the messages too fast?

It is strange because I get it only when I have more than 1 MIDI virtual port

[SpotlightKid]

Both messages happen, when in the event loop in the alsaMidiHandler the call to snd_seq_event_input returns an error, even though the call to snd_seq_event_input_pending before indicated that there are events in the input buffer.

I guess what happens is that the input FIFO overruns, so that events get overwritten, which have not been retrieved yet. For details we would need to look at the ALSA source code.

[SpotlightKid]

Closing this a s "wontfix" now, since using MIDI as such high speeds is out scope for this project. If you want to propose a fix, feel free to submit a PR and re-open this issue.