Assistant capability and reverse-engineering thereof

[tucnak]

I have had this idea for some time, but have very little information to do away with.

Thankfully, Pixel Buds Pro work well with GrapheneOS on Android, so I had been wondering: you know how these support AI assist via Google Assistant application? Now that there's many Mac Studio's and Ryzen DDR5-based inference environments in the market, on-premise cloud deployments have become more commonplace. Also: there has been improvements on-edge, namely whisper.cpp and llama.cpp builds via Android NDK are able to leverage local accelerators.

Is there a chance that the proprietary AI assist capability from Google isn't entirely proprietary?

I wonder if AI assist is ultimately implemented using some signalling protocol over Bluetooth that basically defers to recording and subsequent playback of arbitrary audio. Obviously, there's playback interrupt features but it's probably handled by the firmware itself. If this is the case, then we could potentially modify Pixel Buds application itself to use alternative inference providers.

Based on what you know about Bluetooth & Pixel Buds specifically, how hard could it be, and where would we want to start to learn specifically what's happening under the hood? If it should involve reverse-engineering the relevant apk's, would it also be tractable? If yes, what are the chances it would yield readable, meaningful results? I reckon they probably use NDK themselves, tapping into Android's bluetooth internals, and from the symbols & sniffing alone would it be possible to infer complete logic?

P.S. Great work, your code is quite tidy & indeed, would be worthwhile to investigate how what you do relates to what they do.