Airplay and (Relay) Remotes

[systemcrash]

What do you need help with?

Tja @postlund ! I'm working on the OpenAirplay AP2 receiver , and trying to figure out a few details of how Remotes work just for Airplay (audio). We don't use pyatv... yet. It would probably work, but seems like a sledgehammer to crack a nut.

I've got HK implemented, and threading working, so iPhones can independently change properties of an active AP2 receiver.

Something like this happens:

Device 1 connects, and sends its groupUUID, which the receiver ZeroConf broadcasts. Then all the flies come for the shit banquet :smiling_imp:

The gid flag present in mDNS is what triggers other devices to present the "Control Other TVs & Devices" in e.g. Music app as long as the receiver broadcasts flag bit 17 for RelayRemote. The other devices connect, present a type 130 stream, and poll the receiver for what's going on. And differing somewhat from your description in the docs, they connect to the dataPort, but never do anything more, similar to how you describe the eventPort - nothing happens there. I tried passing it to a HAPSocket and encrypting with the Events-Salt etc you wrote about. Maybe I'm missing some special sauce there, but the senders don't send seed or other keys like your docs mention. It does RECORD, then on SETUP they send e.g.:

{'streams': [{'channelID': 'DF40296C-7924-42EE-B401-FB209A52EAEA', 'clientTypeUUID': '1910A70F-DBC0-4242-AF95-115DB30604E1', 'clientUUID': '2476F28A-BF17-4631-B267-A9482E892402', 'controlType': 1, 'type': 130}]}

They use controlType 1 because I broadcast the stats flag RemoteControlRelay = 1 << 11. Type 2 is Direct, I think.

So at this point, Remote devices connect via HK pairings, poll, and disconnect after three 'attempts' to do what they try to do, either triggered via the Music app, or via HomeKit. But no device tries to encrypt the dataPort or connect to anything else. Am I missing something or is everything like this on iOS <= 15? It feels like I have not yet added something necessary in a response to SETUP or RECORD (but those exchanges look largely as written in your docs).

Do you have any pointers, or ideas what's going on here? This is AP2 for audio specifically. But the underlying details should be the same...

If it's any consolation, I get the same behaviour with Sonos speakers. :man_shrugging: Maybe it really is supposed to be like this, but that's a bit shit.

[postlund]

Hey! Interesting findings but I'm not sure I can help much I'm afraid (but your findings will definitely by interesting) as I have only looked at the MRP-tunnel support, so I'm not sure how the data channel is supposed to work when dealing with just audio. I know that wantsDedicatedSocket must be present (and set to True) when setting up the stream, otherwise I don't even get a dataPort. For MRP, the client must also send a DeviceInfoMessage on the data channel before anything happens at all. I'm not sure what is actually sent over the data channel in your case, did you ever sniff that so you have any reference?

[postlund]

Btw, you seem to know your way around the various settings and properties... Do you know the proper way to distinguish an AirPlay v1 receiver from an AirPlay v2 receiver? I currently don't support AirPlay v2 receivers for audio streaming (other than Apple devices which supports both versions), so I would like to exclude such devices from my scan results.

[systemcrash]

Hey! Interesting findings but I'm not sure I can help much I'm afraid (but your findings will definitely by interesting) as I have only looked at the MRP-tunnel support, so I'm not sure how the data channel is supposed to work when dealing with just audio. I know that wantsDedicatedSocket must be present (and set to True) when setting up the stream, otherwise I don't even get a dataPort. For MRP, the client must also send a DeviceInfoMessage on the data channel before anything happens at all. I'm not sure what is actually sent over the data channel in your case, did you ever sniff that so you have any reference?

wantsDedicatedSocket must be present (and set to True) is... not always true for me. In fact, I never see wantsDedicatedSocket. I wonder whether you could test for me: you have an iPhone and a receiver which both support AP2? Seeing what differences you get could be interesting. You can try my dev branch (run it with e.g. python3 ap2-receiver.py -n en0 --debug to see whether you get the same results when you:

• add the receiver in HK
• cast to it via AP, and
• try to remote control it from HK from a separate device (tho your main iPhone might also try to set up an RC session)

I respond with a dynamic dataPort, but nothing yet is ever set up. I wonder whether it falls back to one of the older static ports, but captures don't seem to indicate anything like that. Maybe I missed something.

Do you know the proper way to distinguish an AirPlay v1 receiver from an AirPlay v2 receiver?

Hmm - the best way for receivers, I think, is certain bit-flags. PTP is the big one: airplay 2 mandates it. Everything before then is NTP (bit-flag 45). So if the PTP bit-flag 41 is off, it's likely not AP2. If I turn off PTP, all clients try AP1 ANNOUNCE method. On, and they don't. Those bit-flags are usually mutex, but it's possible to enable both and advertise support for older and newer clients. There are other more subtle tests like:

• RTSP port numbers (7000 for AP2)
• whether all of the mDNS flags are RAOP, and two characters "XX"
• Probably also HK pairing support - bit-flag 46 and 48
• You might also want to look out for the marketing version string version and SDK version sdk which are sometimes advertised.

[systemcrash]

Bit-flag 30 also seems to partition behaviour. sourceVersion or srcvers also seems like a big decider of certain features. The line appears to be around 355. Above, PTP, below, NTP.

[postlund]

wantsDedicatedSocket must be present (and set to True) is... not always true for me. In fact, I never see wantsDedicatedSocket. I wonder whether you could test for me: you have an iPhone and a receiver which both support AP2? Seeing what differences you get could be interesting. You can try my dev branch (run it with e.g. python3 ap2-receiver.py -n en0 --debug to see whether you get the same results when you:

• add the receiver in HK
• cast to it via AP, and
• try to remote control it from HK from a separate device (tho your main iPhone might also try to set up an RC session)

I believe this to be an Apple-specific setting that is not part of the general AirPlay 2 specification, but I can of course be wrong. There is likely a flag that indicates if this kind of remote is possible to set up, but I'm not sure. Did you try to replicate the feature flags of a HomePod for instance to see how iOS behaves? My HomePod announces 0x4A7FCA00,0xBC354BD0 as a reference.

I respond with a dynamic dataPort, but nothing yet is ever set up. I wonder whether it falls back to one of the older static ports, but captures don't seem to indicate anything like that. Maybe I missed something.

Hard to tell, I don't have any ideas I'm afraid.

Do you know the proper way to distinguish an AirPlay v1 receiver from an AirPlay v2 receiver?

Hmm - the best way for receivers, I think, is certain bit-flags. PTP is the big one: airplay 2 mandates it. Everything before then is NTP (bit-flag 45). So if the PTP bit-flag 41 is off, it's likely not AP2. If I turn off PTP, all clients try AP1 ANNOUNCE method. On, and they don't. Those bit-flags are usually mutex, but it's possible to enable both and advertise support for older and newer clients. There are other more subtle tests like:

• RTSP port numbers (7000 for AP2)
• whether all of the mDNS flags are RAOP, and two characters "XX"
• Probably also HK pairing support - bit-flag 46 and 48
• You might also want to look out for the marketing version string version and SDK version sdk which are sometimes advertised.

Ok, great. You haven't found any obvious way either 😉 I believe that is good enough for my filtering at least. Thanks!

[systemcrash]

Welp, I'm SOL. AP2 is sending some weird blob, OPACK(?) which no tools out there seem to be able to decode. Not even the compiled one from your doc. I see a bunch of UUIDs and some strings in there. Just gotta keep hacking away I suppose. Like, how many formats do Apple need to throw into AirPlay?

[postlund]

Can you attach it or send it to me, I can take a look.

[systemcrash]

Verbatim copy from python: b'\xbc\x02\x08\x0f\x12$28E1F94D-7FAB-492F-9C18-593DF5B38C52 \x00\xa2\x01\xe7\x01\n$ADAF1D69-4375-4642-BA64-739781FFEE3F\x12\x06iPhone\x1a\x06iPhone"\x0618E212*\x16com.apple.mediaremoted8\x01@lH\x01P\x01b\x0fcom.apple.Musich\x01p\x01\x88\x01\x02\xa2\x01\x1170:70:0d:5e:22:81\xa8\x01\x01\xb0\x01\x01\xc0\x01\x01\xe8\x01\x01\xf0\x01\x00\xfa\x01\x12com.apple.podcasts\x82\x02$8463668E-DD75-4789-A13F-C6DB076128FC\xa8\x02\x00\xb0\x02\x01\xba\x02\tiPhone9,3\xaa\x05$836C6DC2-CB9C-4C5C-BB7E-8B7565C4AFE2'

It's the chunk which comes out of this, in a POST: /command application/x-apple-binary-plist

{'params': {'data': b'\xbc\x02\x08\x0f\x12$29FA41FF-B6EB-4A95-A0E6-C2EBAD49E831 \x00\xa2\x01\xe7\x01\n$ADAF1D69-4375-4642-BA64-739781FFEE3F\x12\x06iPhone\x1a\x06iPhone"\x0618E212*\x16com.apple.mediaremoted8\x01@lH\x01P\x01b\x0fcom.apple.Musich\x01p\x01\x88\x01\x02\xa2\x01\x1170:70:0d:5e:22:81\xa8\x01\x01\xb0\x01\x01\xc0\x01\x01\xe8\x01\x01\xf0\x01\x00\xfa\x01\x12com.apple.podcasts\x82\x02$8463668E-DD75-4789-A13F-C6DB076128FC\xa8\x02\x00\xb0\x02\x01\xba\x02\tiPhone9,3\xaa\x05$1736D4BC-859B-4E43-9286-CF93C94FB3EB'}}

Maybe it's not OPACK, but one of those other protobuf

[postlund]

It's a variant and a protobuf message, I.e. MRP, which is exactly what you are after. Congratulations! 🎉 See here:

https://github.com/postlund/pyatv/blob/e8ae99be766370ef36543210105abe8c2dbcf611/pyatv/protocols/airplay/channels.py#L126

[systemcrash]

Bangin. I figured it was something easy. Break time :facepalm: Thanks.

[systemcrash]

Hmm, now what to do with it all. Any tips on how I can import your protobuf implementation without lifting in the other deps? Or they're all needed at some point depending on how the remote is feeling?

[postlund]

You basically want to implement the MRP service in my fake device, which is available here:

https://github.com/postlund/pyatv/blob/master/tests/fake_device/mrp.py

It is extremely hacky and specific to test pyatv. It is also not compatible with iOS as-is as I haven't made any integration towards iOS. So you need to make adjustments.

You don't need all dependencies, but the protobuf messages are necessary. Ideally those should be a separate python package, but I haven't seen any need for that so far so didn't put any time into that. You will have to copy them I guess.

[postlund]

Btw, what was the solution to trigger iOS to open a remote control channel? I'm very interested in the same check I pyatv, to know if I should set up a remote control channel or not.

[systemcrash]

AP has various sets of flags, and there's one called status flags. There are about 17 bits I know about.

https://github.com/openairplay/airplay2-receiver/blob/ca4db5c31c20edd03fb2e7ca1d521c02bb1cb15d/ap2/bitflags.py#L196

Flip 11 (and possibly also 17) to on and devices attempt connection no matter what - so the flag is intended to be broadcast as on when a device is playing something. But that's only for a relay type remote connection. Not for a direct. I don't know how far or many those flags are, so play around. At least up to 32.

At playback, I XOR that bit on and broadcast the flags, and all the devices on the same network want to know about my receiver :smile:

[systemcrash]

I wonder how easy it is to proxy to airplay (audio only) targets? I tried last night and the proxy tools give up because the targets are not TVs. Proxying would accelerate my remote/relay hacking efforts. I see in another issue (1255) you made some progress, but nothing well documented.

[systemcrash]

BTW did a bit of digging, and it seems these are the remote IDs:

    MEDIA_REMOTE = '1910A70F-DBC0-4242-AF95-115DB30604E1'
    VALERIA = '2B6B4700-D998-4081-89F7-5D9AF93846E2'
    APSYNC = '8186BE43-A39A-4C42-9D0E-60BDB9CE1FE3'
    MATCHPOINT = 'A6B27562-B43A-4F2D-B75F-82391E250194'

You might have already figured those out from the protobufs...