Use iDrive Knob to send keyboard commands within Screen Mirroring for Android Auto on casting phone

[r3dalk]

Hold your horses, I know the quick answer is AA is not supported, but I have been fiddling around with some things.

First of all, this is my first time opening an issue on a open/pubic project so apologies in advance for any mistakes. Also, your app literally saved me dropping an absurd amount of money on an iPhone, so I'm really, really thankful for that!

I'll detail what I've tried and achieved, and then explain the issue/request I'm making, to understand wether it's achievable (and I'm hoping it is).

I have a 2021 BMW 2 Series (F22) with what is reported through AAiDrive as a NBTevo_ID5, and have been using the screen_mirror plugin since I got the car to stream Waze, running the AAiDrive app on a Samsung S23 Ultra with the screen_mirror addon, always through Bluetooth.

I have setup multiple Samsung routines that somewhat make the experience more usable (autorotate to landscape Waze if car's BT is detected, night/day mode if BT and time of day, auto-tap the screen to allow the screen mirroring permissions), and then I tried the next step: I got an old Android phone, installed the AAiDrive, and ran Android Head Unit Remote to emulate Android Auto on the old phone, then connected it to the car, streaming AA interface on the iDrive and my s23 connected just to the old phone. This worked, but there is no way to input other than touching the old phone's screen, which is impractical. Then, I installed an app on my s23 that emulates a Bluetooth keyboard, and coded 5 buttons that are:

• Left arrow and Right arrow (cycle through current Android Auto active view)
• Shift + left arrow and Shift + Right arrow (cycle through Android Auto active views and side bar options, changing from Waze to Spotify, or selecting the menu and such)
• Enter (enter/click on selected item)

This is a tad more convenient, but there's quite some delay in s23 -> old phone BT -> AA interface -> Screen mirror, which makes it really not usable. I thought, and thus the title of my issue, that maybe there would be a way when Screen_mirror is running in the car to map the free knob movements to those key strokes, particularly in the way which 2018-202 Audi MMI interface knob works:

• Up/down knob: cycle through views (Shift + left arrow and Shift + Right arrow BT key presses)
• Scroll left/right to navigate selected view (cycle through current Android Auto active view)
• Press to enter

I know left and right knob are reserved to go back and into side screen options so I didn't include those. I think (talking in the dark here) if there is a way in which you can detect the knob's pressing/tilting/scrolling (or maybe implementing floating/transparent buttons), the action on pressing those buttons could be sending a Keyboard input to the phone (or the straight command, don't know if that's possible in Android).

I have downloaded and tried to fiddle around with the addon's code using your Wiki on the BMW Connected Apps protocol and some GPT's aid, but I have no Kotlin knowledge (Java, Bash and Python guy) and it's been a while since I last actively coded, so my efforts were short of emabarrassing.

Still, I think it might be feasible encoding/making the car send those commands to the phone, or if it's too much a niche request, I'd be thankful for some guidance as to how to implement it, in case there's some good samaritan around who knows Kotlin and Android development and is willing to try and give the idea a go. I had thought it could be added as an "option" to the addon, like a toggle, to avoid people who use the app just as a mirroring to send unwanted commands.

I'd be more than glad to see the full AA working on the car, and even if it's this way, I think it could provide a fair enough close functionality that would be great having. My next steps in trying, were these inputs from the car be possible, would be to get a Pi Zero W with a BT module to run an AA Emulator, plug it to the car's USB and get an almost-seamless AA Experience on a BMW the way BMW never meant it. I'd be more than glad to do that (I have the skills required for it, unlike the one I'd need for this modification I'm requiring :) )

I may not have worded some things correctly, or may have not gone into enough details where those are needed, but I guess the TLDR of the request is: could be implemented some simple knob-to-phone commands as Keyboard inputs to interact with it, or provide some guidance as to how it could be done?

Please, shall any doubts or discussions arise, don't hesitate asking! I'll do my best to try and get this working. Also, if the idea can be done, I'll be more than happy to document and provide step-by-step details on how to get the "almost AA" experience!! :)