key events not working in Reaper on macOS

[httnn]

the NSView's keyDown/keyUp methods are never getting called in Reaper when there's keyboard input with the plugin window in focus. neither turning on "Send all keyboard input to plugin" or opening the UI in its own window changed the situation.

curiously, the events are still consumed on some level as Reaper is also not receiving keyboard events when the plugin window is in focus (space doesn't toggle playback, for example).

i'm running macOS 13.5.2 and Reaper 6.43 and using baseview with nih_plug

[httnn]

wondering if the problem actually lies within nih_plug: the VST3 onKeyDown implementation returns kResultOk, but according to the API docs it should return either kResultTrue or kResultFalse depending on whether the key is consumed. perhaps kResultOk is interpreted similarly to kResultTrue?

i have a feeling that Reaper might be using these VST3 key handlers more pedantically than most hosts

[httnn]

returning kResultFalse from the VST3 key handlers indeed allows the key events to flow to the plugin (this has been fixed in nih-plug now https://github.com/robbert-vdh/nih-plug/commit/245add6530f1c7d83fea5e23636145d982af4d66)! but now the problem is that no key events are being consumed from Reaper's perspective, so e.g. the spacebar and enter keys will always get passed to Reaper. to fix this, baseview would need to have some way to signal to the VST3 key handlers about when to consume/ignore key events, but not sure what that would require.

this is also an issue with JUCE btw: https://forum.juce.com/t/reaper-stealing-spacebar-keystrokes-in-texteditor/30689/8