Verify compatibility with tvOS 18

[postlund]

What to investigate?

Upgrade to current beta and try it out.

Expected outcome

See if anything needs to be fixed.

[knopp]

Video playback (play_url) seems to fail with 500 at /GET playback-info. Even when ignoring the error playback doesn't start.

[postlund]

When playing video or audio or both?

[knopp]

I tried it with a HLS URL.

atvremote -n "Apple TV 4" --debug play_url=https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_4x3/bipbop_4x3_variant.m3u8

[postlund]

@knopp Could you possibly attach some debug logs I can look at?

[postlund]

I can confirm the issue now as well, not sure why it happens. Will have to investigate further.

[postlund]

I suspect that the endpoint I'm using (/playback-info) is no longer available as that is only for AirPlay 1 and that call should not be made anymore. Not sure yet what to do instead though, will dig into that.

[thiccaxe]

If you change the client version in the companion protocol to a i(pad)os18 or higher version, and subscribe to "NowPlayingInfo" events, in some AppleTV+ titles that send over large packets, there is an extraneous null byte inserted before the beginning of the bplist that is encoded as the value of the dictionary key "NowPlayingInfoKey". Removing this null byte allows everything to be decoded properly.

one of more of the following components fail:

• Opack Decoding
• Decryption (no errors)

If this takes place during decryption, which I doubt, the null byte would have to be inserted after the decryption takes place, and somewhere along the way from the crypto internals to the api surface. If it happened before, there would be a mismatch between the data length and the header, which would lead to a decryption error, likely. Moreover, the extra null byte doesn't appear at a power-of-2 location, which could be a internal crypto block size (i.e byte 63/64/65, or byte 127/128/129). This all points to the fact that the encryption is fine

This means that it is likely that the opack decoder is not "up to the standard". It could also be obfuscation measures from apple.

If someone with a mac wants to test the opack, or replicate this "bug?", t o obtain the "bad" opacks, take a look at the following code:

import pyatv
import asyncio, binascii
import logging
from datetime import datetime
import plistlib
logging.basicConfig(
    level=logging.DEBUG,
    datefmt="%Y-%m-%d %H:%M:%S",
    format="%(asctime)s %(levelname)s [%(name)s]: %(message)s",
)

async def mrec(message):
    print("received message: ", plistlib.loads(message["NowPlayingInfoKey"]))

async def main():
    loop = asyncio.get_running_loop()
    atv = (await pyatv.scan(loop, hosts=["192.168.1.3"]))[0]

    atv.set_credentials(pyatv.Protocol.Companion, "")
    atv = await pyatv.connect(atv, loop)
    api: pyatv.protocols.companion.CompanionAPI
    print(atv.remote_control._interfaces[pyatv.Protocol.Companion])
    api = atv.remote_control._interfaces[pyatv.Protocol.Companion].api
    await api.subscribe_event("NowPlayingInfo")
    api.listen_to("NowPlayingInfo", mrec)

    await atv.remote_control.skip_forward(float("NaN"))
    while True:
        await asyncio.sleep(3)

    atv.close()
if __name__ == "__main__":
    asyncio.run(main())

In the system info packet in protocols/companion/api.py you will need to change the following to reflect a i—18 client

"_sv": "600.41.1", # Software Version (I guess?)

I suppose the hacky patch is to indexof "bplist00", check if there is a null byte before it, then look back a bit more to see if the bytes 0x92,0x93 or 0x94 are present 2, 3, or 4 bytes behind, which would represent bytes encoded in opack, and then remove that null byte.

[thiccaxe]

I think it makes more sense if we look at 0x91 having 2^0 bytes of length, 0x92 having 2^1 bytes of length, 0x93 having 2^2 bytes of length and 0x94 having 2^3. This would make the extra "00" a part of the length, and would explain its presence

Previously, we encountered no opacks longer than 0xFFFF. so, byte data was done as 0x90, 0x91, or 0x92. Now that we are looking at data which is longer than 0xFFFF, it jumps up to four bytes of length instead of three — its not linear. It should be a very easy patch in the code.

[petro-kushchak]

@postlund audio streaming started failing with latest iOS18 beta (and latest pyatv 0.15.0)

Attached logs for

atvremote -n "Studio" --debug play_url=https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_4x3/bipbop_4x3_variant.m3u8:

ios18-homepod-pyatv-logs.txt

[knallle]

I suspect that the endpoint I'm using (/playback-info) is no longer available as that is only for AirPlay 1 and that call should not be made anymore. Not sure yet what to do instead though, will dig into that.

Have you found out anything about this? I would love to help but have no idea how to start.

[postlund]

@knallle I have not made any progress in that matter, no. I would start using atvproxy and see if that yields any new insights.

[knallle]

Understood. Is there any (un)official documentation of the API that could be helpful?

[postlund]

You can find various information about AirPlay 2 by just goggling. You can also check some of my work at https://pyatv.dev/documentation/protocols/ (links to other sources are there as well). Similarly you have some documentation for atvproxy here: https://pyatv.dev/documentation/atvproxy/

[knallle]

@postlund A thought.

If i understand the logic in airplay/player/play_url: we initiate playback and then wait for the file to finish playing by:

1. periodically polling (once per second) the Apple TV for playback information
2. when we detect the playback status changing from "playing" to "not playing" we assume playback has finished.

And 1. breaks since, supposedly, we can no longer GET /playback-info.

However Using atvremote, I can still do

atvremote -n "Vardagsrum" playing

to get the playback status of the Apple TV. I've tried to understand, but not yet fully understood, what takes place under the hood of atvremote playing, but could that mechanism be used to poll the playback status in play_url?

[postlund]

@knallle Yeah, I understand your conundrum. The core of pyatv has a public API as defined in interface.py. This is basically what the user (I.e. developer using the library) interacts with. From a black box perspective, you call a certain method within the API and pyatv will internally figure out what to do. And to understand that part...

Internally, protocol implementations are implemented like plugins. Protocols are instantiated and set up based on the configuration you provide when connecting. A protocol may implement as much or little of the public API (interface.py) as it wants: pyatv will figure out what is supported by each protocol and pick an appropriate implementation depending on what you want. I suggest that you read this page:

https://pyatv.dev/internals/design

You can skip down to *Relaying` if you want, that's the important bit. Hopefully that explains the public interface a bit.

The problem here is the AirPlay streaming implementation (for playing a URL). I would expect that the actual streaming works as intended, but as pyatv queries an AirPlay v1 endpoint when streaming with AirPlay v2, that would be a problem. What we need to do is to find an other way of knowing if streaming has finished or not. Maybe we don't even have to poll? Not sure. But we have to use the AirPlay protocol only to do that.

Just to make it even more clear. Running atvremote playing means that we are asking pyatv to find out what is playing based on a configured protocol. That means that you can steam via AirPlay in one session (connection) and (most likely) ask for playing data from a completely different protocol, e.g, MRP, in another session And we need to consider that no other protocol than AirPlay might be set up, meaning we can't use playing internally in a meaningful way.

[knallle]

@postlund Thank you for the elaborate explanation! Your answer was pretty much in line with what i guessed, but it was worth a shot at a simple solution.

I played around with atvproxy and was a bit overwhelmed by the large amount of information. Would have to dig into it more to be productive. Unfortunately, I won't have opportunity to do that for a few weeks because of family reasons. If the problem remains unsolved when I'm back I will give it another go. Hopefully, someone steps in from the sidelines sooner than that.

[NprogramDev]

@postlund Anything new? Together with @PaddeCraft I experimented with this library a bit, but didn’t have any luck getting it to work either. But we are still learning about the “HTTP/TCP-like” communication protocol and don’t really know how to debug those protocols. For reference we have tried it on multiple AppleTV HDs (4th Gen; A1625), on none of them the playback worked (debug logs from the ˋatvremoteˋ command: here). We have also tried to capture the network-traffic of different AirPlay clients that seem to still work (AirMyPC and AirFlow), but still didn’t get a clue on how the problem could be solved (that could however be caused by our lack of knowledge on the protocol(s)).

[postlund]

@NprogramDev I have not looked into it yet, but I will try to find some time to do that. Kinda important to get it working again. I'll see if I can peek at the messages again and see if i can gain further insights. Sniffing traffic from other clients is pretty fruitless as the relevant traffic is encrypted.

[DennisFury]

Friendly ping, any progress on this? Just noticed play_url isn't working pyatv.exceptions.HttpError: HTTP/1.1 method GET failed with code 500: Internal Server Error

as a work around, I download the video file then use plexapi to play through the plex client, but it would be much easier to play directly from a URL

[oxplot]

Dumping some debug logs here:

=== RTSP REQUEST DEBUG ===
Method: POST
URI: /feedback
Headers: {
  "CSeq": 13,
  "DACP-ID": "E4DE22472A67D002",
  "Active-Remote": 1314732296,
  "Client-Instance": "E4DE22472A67D002"
}
==========================

=== SENDING RTSP REQUEST ===
Method: POST
URI: /feedback
Protocol: RTSP/1.0
CSeq: 13
Remote IP: 192.168.2.207
Local IP: 192.168.2.61
============================
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Sending RTSP/1.0 message: b'POST /feedback RTSP/1.0\r\nUser-Agent: AirPlay/550.10\r\nCSeq: 13\r\nDACP-ID: E4DE22472A67D002\r\nActive-Remote: 1314732296\r\nClient-Instance: E4DE22472A67D002\r\n\r\n'
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Received: b'RTSP/1.0 200 OK\r\nDate: Wed, 16 Jul 2025 03:15:03 GMT\r\nContent-Length: 55\r\nContent-Type: application/x-apple-binary-plist\r\nServer: AirTunes/890.68.4\r\nCSeq: 13\r\nX-Apple-ProcessingTime: 0\r\nX-Apple-RequestReceivedTimestamp: 495888178\r\n\r\nbplist00\xd1\x01\x02Wstreams\xa0\x08\x0b\x13\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x14'
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Got RTSP response: HttpResponse(protocol='RTSP', version='1.0', code=200, message='OK', headers={'Date': 'Wed, 16 Jul 2025 03:15:03 GMT', 'Content-Length': '55', 'Content-Type': 'application/x-apple-binary-plist', 'Server': 'AirTunes/890.68.4', 'CSeq': '13', 'X-Apple-ProcessingTime': '0', 'X-Apple-RequestReceivedTimestamp': '495888178'}, body=b'bplist00\xd1\x01\x02Wstreams\xa0\x08\x0b\x13\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x14'):

=== RTSP RESPONSE DEBUG ===
Status code: 200
Headers: {'Date': 'Wed, 16 Jul 2025 03:15:03 GMT', 'Content-Length': '55', 'Content-Type': 'application/x-apple-binary-plist', 'Server': 'AirTunes/890.68.4', 'CSeq': '13', 'X-Apple-ProcessingTime': '0', 'X-Apple-RequestReceivedTimestamp': '495888178'}
Body: b'bplist00\xd1\x01\x02Wstreams\xa0\x08\x0b\x13\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x14'
===========================
Feedback response received successfully
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Sending HTTP/1.1 message: b'GET /playback-info HTTP/1.1\r\nUser-Agent: pyatv/0.16.1\r\n\r\n'
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Received: b'HTTP/1.1 500 Internal Server Error\r\nContent-Length: 0\r\nServer: AirTunes/890.68.4\r\nX-Apple-ProcessingTime: 0\r\nX-Apple-RequestReceivedTimestamp: 495888229\r\n\r\n'
2025-07-16 03:15:03 DEBUG [pyatv.support.http]: Got HTTP response: HttpResponse(protocol='HTTP', version='1.1', code=500, message='Internal Server Error', headers={'Content-Length': '0', 'Server': 'AirTunes/890.68.4', 'X-Apple-ProcessingTime': '0', 'X-Apple-RequestReceivedTimestamp': '495888229'}, body=''):
2025-07-16 03:15:03 DEBUG [pyatv.protocols.airplay.player]: HTTP error getting playback info: HTTP/1.1 method GET failed with code 500: Internal Server Error (code: 500)
2025-07-16 03:15:03 DEBUG [pyatv.protocols.airplay.player]: Attempts remaining: 2
2025-07-16 03:15:04 DEBUG [pyatv.support.http]: Sending HTTP/1.1 message: b'GET /playback-info HTTP/1.1\r\nUser-Agent: pyatv/0.16.1\r\n\r\n'
2025-07-16 03:15:04 DEBUG [pyatv.support.http]: Received: b'HTTP/1.1 500 Internal Server Error\r\nContent-Length: 0\r\nServer: AirTunes/890.68.4\r\nX-Apple-ProcessingTime: 0\r\nX-Apple-RequestReceivedTimestamp: 495889232\r\n\r\n'
2025-07-16 03:15:04 DEBUG [pyatv.support.http]: Got HTTP response: HttpResponse(protocol='HTTP', version='1.1', code=500, message='Internal Server Error', headers={'Content-Length': '0', 'Server': 'AirTunes/890.68.4', 'X-Apple-ProcessingTime': '0', 'X-Apple-RequestReceivedTimestamp': '495889232'}, body=''):
2025-07-16 03:15:04 DEBUG [pyatv.protocols.airplay.player]: HTTP error getting playback info: HTTP/1.1 method GET failed with code 500: Internal Server Error (code: 500)
2025-07-16 03:15:04 DEBUG [pyatv.protocols.airplay.player]: Attempts remaining: 1
2025-07-16 03:15:05 DEBUG [pyatv.core.protocol]: Sending periodic heartbeat 6 (AirPlay:192.168.2.207)

=== FEEDBACK REQUEST DEBUG ===
Sending feedback to Apple TV at 192.168.2.207
===============================