iina
Ctrl-s (screenshot window) does not work
System and IINA version:
- macOS 13.6.3
- IINA 1.3.4
Expected behavior:
Using the mpv Default key bindings and pressing the ⌃s key takes a screenshot of IINA's window.
Actual behavior: Nothing happens when the ⌃s key is pressed. No screenshot is generated.
Checking the mpv log shows the screenshot window command is failing:
[ 34.592][d][cplayer] Run command: screenshot, flags=73, args=[flags="window", legacy="unused"]
[ 34.630][e][cplayer] Taking screenshot failed.
Steps to reproduce:
- Configure IINA key bindings to use
mpv Default - Start playing a video
- Press ⌃s
- [x] MPV does not have this problem.
How often does this happen? Every time.
From the entry for the screenshot command in the mpv manual:
screenshot <flags> Take a screenshot.
Multiple flags are available (some can be combined with +): … <window> Save the contents of the mpv window. Typically scaled, with OSD and subtitles. The exact behavior depends on the selected video output.
The expectation is that this form of the screenshot command is somehow tied to the mpv UI and thus can not be used by a libmpv client. That needs to be verified with the mpv project.
For now the ⌃s key binding should be removed from the mpv Default key bindings.
If the window flag is indeed not usable by a libmpv client then IINA may be able to implement this feature itself.
I wonder if screenshot window should really be allowed as a libmpv command.
Side comment / rant. IINA could replicate this by taking a snapshot of its own player window. I have working code which calls CGWindowListCreateImage to create a CGImage from the window's contentView. But Apple has created a lot of uncertainty here.
CGWindowListCreateImage is, as best I can tell, the standard way to convert a view's contents to picture data. But this API was deprecated in Sonoma with a note stating that ScreenCaptureKit should be used instead. I wonder if whoever at Apple decided to deprecate the original API actually realizes how drag & drop previews have been generated on a Mac.
But ok, I played with ScreenCaptureKit. I discovered there is no way to take a screenshot (or any of its APIs actually) without requiring the user to go to System Settings and allow IINA "to record this computer's screen and audio". This means that if CGWindowListCreateImage is not used, I don't see another way for IINA to create images from its own views without asking for permission to record the whole screen and access the microphone.
Not just that, but they are presently allowing both APIs to live side by side, and only one of them needs permission to work. This is fake security. I wonder if this idea came from the same people who decided to require a special permission for access to ~/Downloads (which is why I now don't use that folder at all, and instead use ~/ActualDownloads for all my downloads).
Yes, this is why I think the current solution for now should be to remove the key binding to the screenshot window command. There are more important issues to address. I created this issue so it wouldn't be totally forgotten.
The first place to start would be to explore what mpv is doing and understand why it is failing. If indeed the solution requires IINA to implement then yes it seemed like that would require using CGWindowListCreateImage and ScreenCaptureKit.
I found people discussing the security prompts. Apparently that was added for CGWindowListCreateImage as well in a macOS Ventura Beta. Maybe Apple backed off that due to negative feedback? I agree with ScreenCaptureKit requiring permission be granted this seems like inconsistent security practices.
If you do want to address this now, that is fine with me.
If you do want to address this now, that is fine with me.
No, I agree that this is not worth bothering with right now. It's just an annoyance. If people really want to take a screenshot of the window, they can just use ⇧⌘5.
I found people discussing the security prompts. Apparently that was added for CGWindowListCreateImage as well in a macOS Ventura Beta. Maybe Apple backed off that due to negative feedback?
Could be. If they wanted to modify its behavior to match ScreenCaptureKit's, they'd have to make it into an async API, because all of its APIs need to wait for the user to respond to the permissions prompt. This would be infeasible to retrofit for CGWindowListCreateImage. They could have changed the legacy API so it returns all black pixels until the user grants permissions. Possibly some apps would still break. It is a curiosity and maybe an insight into Apple's priorities.
I generally think that taking a screenshot of the window is an unusual operation. Is this actually what is desired here? Or is this a feature request for the current video frame, which I am sure there is better API to grab an image for?
I entered this issue to "kick the can down the road" as it involves work to first understand exactly what this mpv command does and why it is failing. I'm thinking this is a very low priority issue and we should not work on it at this time, but I thought we should at least have a record of the problem.
This came up due to a regression with the mpv screenshot command. In fixing that we took a close look at IINA's behavior and found some other existing problems that we are fixing.
I just played with mpv and the screenshot command with the various flags. One difference is that the resulting screenshot with the window flag is scaled, whereas the other flags use the original resolution.
The IINA mpv Defaults key bindings correctly reflects mpv and binds ⌃s to screenshot window, which is failing as discussed above. So my proposal is what we remove this binding for now and take another look at this in the far future.
I see we have a priority: low label. I will apply that.
Window screenshot works fine with libmpv clients as well. I think the issue is that you don't have MPV_RENDER_PARAM_ADVANCED_CONTROL set which is necessary for gpu screenshots. The distinction is a bit annoying, it used to be the case in some ancient mpv version that screenshots were software rendered while GPU screenshots were simply grabbed from the framebuffer (and hence would be taken after scaling, with osd, etc.). Later this limitation was reduced, and now non-window screenshots can go through the gpu rendering path as well, so I think the only difference with window screenshots is that they explicitly rendered onto an unscaled fbo, without osd.
https://github.com/mpv-player/mpv/blob/master/video/out/gpu/video.c#L3437-L3458
But with libmpv clients setting MPV_RENDER_PARAM_ADVANCED_CONTROL is necessary for gpu screenshots. So if it's disabled then only sw scaled non-window screenshots will work, while window screenshots will not.
Btw blame shows https://github.com/iina/iina/commit/e0dc0bf577c3b954d2167015e7b8caf05fdca83b. Don't know why was originally disabled, there's no harm in having it enabled.
@krackers Thank you very much for pointing this out!
At one point I investigated that commented out code that sets MPV_RENDER_PARAM_ADVANCED_CONTROL. I too don't know why it was disabled, but I've always assumed it was to enable support for non-fatal timeouts to ensure the mpv core does not deadlock forever as per this comment in render.h:
Basically, the situation that your render thread waits for a "not safe" libmpv API function to return must not happen. If you ignore this requirement, deadlocks can happen, which are made non-fatal with timeouts; then playback quality will be degraded, and the message mpv_render_context_render() not being called or stuck. is logged. If you set MPV_RENDER_PARAM_ADVANCED_CONTROL, you promise that this won't happen, and must absolutely guarantee it, or a real deadlock will freeze the mpv core thread forever.
I probably read this comment:
Rendering screenshots with the GPU API if supported by the backend (instead of using a suboptimal software fallback via libswscale).
But that makes it sound like mpv will fall back to software, not fail.
What else is IINA missing out on by not enabling MPV_RENDER_PARAM_ADVANCED_CONTROL?
Quite a long time ago I tried running with MPV_RENDER_PARAM_ADVANCED_CONTROL enabled to check for deadlocks and did not manage to trigger one. Of course that was not a deterministic test.
IINA will sometimes trigger that warning. I have seen it during shutdown.
I ran a quick test with MPV_RENDER_PARAM_ADVANCED_CONTROL enabled and screenshot window worked correctly.
I'm thinking if IINA were to consider enabling this we should wait for the beta release as the next release is intended to fix regressions.
But that makes it sound like mpv will fall back to software, not fail.
That's only possible for non-window screenshot though, where it passes the mp_image through libswscale
it was to enable support for non-fatal timeouts to ensure the mpv core does not
Haven't looked too much at the DR codepaths but my understanding is that the two features that setting this flag enables for libmpv clients are direct rendering and gpu screenshots. Direct decoding (zero copy decoding) doesn't matter on mac because it's version of opengl does not support it anyway, best you can do is opengl-pbo. And for most people it also doesn't matter since they're using hwdec.
enabled to check for deadlocks and did not manage to trigger one.
Deadlocks should not really be an issue even with ADVANCED_CONTROL enabled, if you're not doing direct rendering. The only addition case it adds is gpu path for screenshots, but that's exactly the same as just a normal render. So as long as you do it in the same dispatch queue as you normally render, then there won't be an issue.
IINA will sometimes trigger that warning. I have seen it during shutdown.
Yes, it can happen during shutdown if you don't coordinate sequence of events carefully. What happens is that mpv sends shutdown message to VO, but VO can only process that shutdown message after it exists the flip loop. But the flip loop is itself blocked until the vsync is received from the display link. So you cannot shut down display link until the libmpv uninit is fully finished. (Actually this is still a bit racy, the better solution is to modify libmpv so that as soon as you shutdown it no longer waits for the usual render loop).
Actually in your case maybe this is the issue
videoLayer.suspend()
player.mpv.mpvUninitRendering()
You are suspend the render dispatch queue before uninit. But this means that if VO thread is blocked on waiting for the update, it can only service the shutdown after the timeout. I think you should remove videoLayer.suspend().
@krackers Just noticed your additional replies. As you can see I was off preparing a PR that enabled advanced control. I will create a separate issue for the shutdown sequence problem that is sometimes triggering the mpv_render_context_render not being called warning and prepare a fix for that as well.
Thanks for pointing out the root cause of that warning!
I asked the other developers why advanced control had been disabled. The answer is that it is the fix for issue #2149, a hang.
Reading through that issue I'm worried there are still problems with how IINA is using the mpv render API methods. I will need to investigate some more.
@low-batt Thanks for digging that up. I'm a bit confused though:
The situation where using advanced_control would deadlock seems to be something like the following:
- You call
mpv_get_propertyfrom the render thread, for a property likevo-passes - The
vo-passesproperty requires a blocking voctrl down to libmpv, which schedules a task on the internal libmpv dispatch queue (note that when I use the term "libmpv dispatch queue" I mean mpv's own implementation of a task queue, NOT GCD dispatch queue), which is to be processed by the render thread during the next call to render update, and the caller blocks until this is processed - But that render update call never comes (because render thread is itself blocked on our
get_propertycall). Deadlock.
This is what wm4 meant by
Note how enabling advanced rendering control disallows the API user from calling certain "blocking" functions from the same thread as the rendering functions.
If you look at the implementation there are only 3 things that can cause a task to be scheduled on the internal libmpv dispatch queue: screenshots, perf stats, and direct-rendering (the last of which is actually unsupported on osx). So the set of things that can cause a deadlock is fairly bounded (because conceptually the only difference in internal state between advanced_rendering enabled/disabled is whether something is scheduled and someone is synchronously waiting on the libmpv dispatch queue).
But this is actually not the issue that IINA faced in the bug you linked. If you look there, you don't even see any mpv_render_* calls, and in fact there's no com.colliderli.iina.mpvgl GCD queue present in the stack trace at all!
Instead, the situation is actually more subtle: we can see in the process dump (thank you to author for including that)
2439 Thread_16655846: mpv/mpv core
+ 2439 get_buffer2_direct (in libmpv.dylib) + 694 [0x1094aa346]
+ 2439 dr_helper_get_image (in libmpv.dylib) + 53 [0x1094b9c15]
+ 2439 mp_dispatch_run (in libmpv.dylib) + 91 [0x10944e72b]
+ 2439 _pthread_cond_wait (in libsystem_pthread.dylib) + 724 [0x7fff71e135cb]
+ 2439 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff71d5a1b2]
The core goes to decode a frame (in vd_lavc, calls into libavcodec, which in turns calls back into the registered directRendering callback to get a GPU buffer. (Unrelated: on osx OpenGL is old and does not support AZDO functions, so this always returns null. But nonetheless the callback still happens every frame when software decoding. I mistakenly thought that Direct Rendering callback would never be called if it was unsupported, but it seems it still is, and it fails later on.)
This is where the issue lies: because IINA is not calling mpv_render_update at all, the core remains blocked. So naturally all other calls into the core block as well.
Basically if VO is active, IINA needs to process the dispatch queue, which for some reason it doesn't seem to be doing in this edge-case (I have not looked into IINA code, I remember there was some calls to suspend() or something which needs to be investigated. The fact that it happens when one file ends and another file begins is suspicious).
Edit:
For some reason it only happens when the window is offscreen.
Maybe this is part of it, does IINA have some logic to skip calls if the window is offscreen? Or perhaps it's osx being smart enough to skip the layer draw entirely if it's off screen? Skimmed IINA codebase and indeed you are sticking the call to render_update inside canDraw of the layer. Assuming that the queue itself is still active (not suspended) at the time of the deadlock then the only thing I can think of is osx trying to be clever and skipping such calls entirely when window is occluded.
Assuming that the queue itself is still active (not suspended) at the time of the deadlock then the only thing I can think of is osx trying to be clever and skipping such calls entirely when window is occluded.
Yes - my past testing has indicated that this is what happens. Eventually I discovered this is an unpublicized feature called App Nap and you can check its state by showing the App Nap column in Activity Monitor.
Basically if VO is active, IINA needs to process the dispatch queue, which for some reason it doesn't seem to be doing in this edge-case (I have not looked into IINA code, I remember there was some calls to suspend() or something which needs to be investigated. The fact that it happens when one file ends and another file begins is suspicious).
Thank you for this! The need for suspend() had long been confusing me. The code calls this in 4 places: while entering full screen, exiting full screen, when IINA initiates shutdown, and when mpv initiates shutdown and MPV_EVENT_SHUTDOWN was received.
@krackers I think we've talked about the full screen cases; quick rehash: they can be removed but there are other bugs in the full screen transition code which must be fixed too to actually get a clean animation. And unless I'm missing something, the other two cases can only happen while quitting the app. There's no direct call to suspend() when changing tracks, but if the user had the pref fullScreenWhenOpen enabled then the track change may have triggered a call to enter full screen.
I noticed the affected issue is over 4 years old, so I dug up the code from the time of the report. I found that at the time, suspend() was only called for the full screen toggles (i.e. not during shutdown) so those must have been added after the bug report.
The calls to suspend() during shutdown were the only remaining invocations of them in my fork of IINA. I removed them and found no adverse impact. I'd recommend doing the same in this project if there is any potential danger.
As for the calls to suspend() at full screen toggle...sounds like they are unsafe as well? The toggle will suspend for the duration of the animation (0.5 sec). There are some potential tasks which may call get_property from the main queue during that time - the most immediate which comes to my mind is the Inspector window, which can query almost any set of properties and does so once per second while it's open.
@krackers @svobs This discussion is very helpful.
Issue #3997 shows a mpv_get_property related deadlock. A full process sample is available. I believe the problem is that this requirement is being violated:
Basically if VO is active, IINA needs to process the dispatch queue
The mpv queue is processed byMPVController using the com.colliderli.iina.controller queue. In #3997 MPVController is blocked trying to draw in response to a MPV_EVENT_PLAYBACK_RESTART event.
PR #4819 changes MPVController to asynchronously dispatch most work to the main queue. I am proposing this change to fix numerous data races by only accessing the data using the main thread, rather than resorting to locking. The requirement to continue to process mpv events provides another reason to simplify the work performed using the com.colliderli.iina.controller queue.
The next release of IINA will be focused on fixing simple issues. After that I believe we will be starting on the long planned feature release. That will include beta releases, which is a good time to make more risky changes such has fixing thread issues. That is why I had been delaying posting the changes in PR #4819 until recently when I started to prepare PRs for the feature release.
There are some other unexplained hangs. I will look for them and continue to think about this issue.
@svobs
unpublicized feature called App Nap
Well app nap itself is fairly well known, but I'm surprised the app is scheduled for naptime when audio was playing just a few milliseconds ago. But yeah that could definitely cause it, I also wouldn't be surprised if osx had some other optimizations when windows are off screen (for instance I've observed that when a window is occluded with mpv paused, cvdisplaylink callbacks become less frequent. But if video is playing when occluded this doesn't happen, so something takes into account if there was recent video activity when deciding to deprioritize the app).
The need for suspend() had long been confusing me.
I'm of the opinion that all uses of suspending the render GCD queue should be investigated and removed, because it's too risky to use correctly with libmpv. For shutdown the issues with current iina implementation are already documented on another issue.
I'm not too familiar with the full-screen issue though (I vaguely remember something about it in one of the PRs which ended up taking a tangent into a discussion the nuances of caopengllayer on mac...), could you remind me again why IINA has to suspend it during the FS animation? What happens if you just keep rendering to the layer while the fs animation is in progress? I'd think (or hope) that apple would have made it do something sensible.
As for the calls to suspend() at full screen toggle...sounds like they are unsafe as well
It's "safe" in the sense that it won't cause a deadlock assuming you can guarantee that it's unsuspend in a bounded time, but it can block the entire core for 0.5 seconds which is probably bad for other reasons (it can e.g. lead to audio artifacts if the audio buffer runs dry during that time).
@low-batt
I'm not too familiar with IINA internals, so sorry for noob question but in your linked issue https://github.com/iina/iina/issues/3997 why are both com.colliderli.iina.controller and com.colliderli.iina.mpvgl trying to call into mpv render? I thought all rendering was done through the view layer would go through com.colliderli.iina.mpvgl .
Questions are always fine. Exchanging information is how we get to the root cause of some of these harder problems. I'm always happy to answer questions. Trouble is I am still somewhat a noob and do not know the reasons behind why the code is the way it is.
I can confirm that drawing is not always done by the thread associated with the com.colliderli.iina.mpvgl queue. One example in MainWindowController can be seen here where the main thread is used:
// force rerender a frame
videoView.videoLayer.mpvGLQueue.async {
DispatchQueue.main.sync {
self.videoView.videoLayer.draw()
}
}
I'm not sure why that code is using both the mpvGLQueue and the main queue. There are two other places in MainWindowController that also draw using the main thread.
As long as there isn't some reason why so much work is done directly by the com.colliderli.iina.controller queue and the changes in PR #4819 are approved by the other developers, that would at least remove drawing by that queue.
@krackers:
Well app nap itself is fairly well known, but I'm surprised the app is scheduled for naptime when audio was playing just a few milliseconds ago. But yeah that could definitely cause it, I also wouldn't be surprised if osx had some other optimizations when windows are off screen (for instance I've observed that when a window is occluded with mpv paused, cvdisplaylink callbacks become less frequent. But if video is playing when occluded this doesn't happen, so something takes into account if there was recent video activity when deciding to deprioritize the app).
Well. I took some peeks at the App Nap status of my various apps over the past day to see if I could see some patterns. It looks like it is only ever enabled for "traditional" apps with windows (i.e. not daemons, menu bar apps, etc.). Among Apple's stock apps, Xcode, Notes, and Preview take naps, but looks like [i]Messages does not, oddly. But I've also been running Spotify paused in the background and haven't touched it since yesterday, but it is not napping. This suggest there must be a way for apps to disable it or reject it.
I know that some time ago, @low-batt did a bunch of work getting idle usage down to 0%, so IINA doesn't even need this feature. I'll try to find time to dig up more info.
I'm not too familiar with the full-screen issue though (I vaguely remember something about it in one of the PRs which ended up taking a tangent into a discussion the nuances of caopengllayer on mac...), could you remind me again why IINA has to suspend it during the FS animation? What happens if you just keep rendering to the layer while the fs animation is in progress? I'd think (or hope) that apple would have made it do something sensible.
See this comment, but the gist is that it was done to hide a flickering effect when transitioning into legacy full screen because the window style changes couldn't be applied cleanly while doing a zoom animation. Though it looks like Apple has been making significant improvements to this area over the last few releases, there are a lot of other problematic things (problems even?) with the full screen transition code such that the observed behavior may be unpredictable for different MacOS versions. IMHO it needs a big refactor to do properly. Which I did on my fork. But big changes are not being approved right now.
I thought all rendering was done through the view layer would go through com.colliderli.iina.mpvgl .
That is the normal way to render while playing, yes. But there are also "forced" renders which exist as a workaround when the video is paused. When paused, applying certain changes to the enclosing window such as minimizing + unminimizing or changing the window style for legacy full screen (as mentioned above) will cause the video to go black. It's as though (I'm speculating)CAOpenGLLayer has the only reference to the image and does not preserve it when it loses visibility. But the behavior is very consistent and predictable, and adding a redundant render call brings it back before the user can even see it's gone.
I've mused about finding another way to cache or capture the paused image so that the "forced" renders wouldn't be needed. But never got around to it.
@svobs
way for apps to disable it
Yeah I think there's a way for apps to opt out of it.
transitioning into legacy full screen because the window style changes couldn't be applied cleanly while doing a zoom animation
Curious, is it IINA itself that creates the zoom-animation here? If you normally change the style mask on a window the changes take effect relatively instantly.
has the only reference to the image and does not preserve it when it loses visibility
Yeah that makes sense, probably things like switching style mask cause the window backing store to be flushed and recreated entirely. But why can't we just queue these redundant renders on the existing gl queue, instead of needing to do it from another queue?
Curious, is it IINA itself that creates the zoom-animation here?
Correct. Not sure what the default one would look like, to be honest. I suspect it wouldn't be as pretty as the current one.
If you normally change the style mask on a window the changes take effect relatively instantly.
For most style bits you are correct, but adding or removing the titled style seems to be a special case which has many side effects. For example, when it's removed, Zoom and Minimize menu items are disabled (+ others), and the window loses its entry in the Window menu and Dock. But as recently as Big Sur it also was monstrously slow to toggle this style - perhaps 0.25 sec on its own.
With Sonoma it's effectively instantaneous in some cases, but in others I still can't avoid a small flicker wherein the window seems to momentarily disappear. Compounding the problem is the way the Cocoa does animations. Depending on what else is going on in the window or in other apps, or the CPU is under load, they can sometimes act laggy or show some tearing and it can be really hard to nail everything down. And when using legacy full screen, the menu bar and Dock visibility need to be toggled via a separate call from the zoom and from the style change, and all this is supposed to be look simultaneous.
But why can't we just queue these redundant renders on the existing gl queue, instead of needing to do it from another queue?
Not sure :) They definitely need to be done quickly to avoid a visible flash of black. On the other hand, the queue should be empty and ready for work. I'll try playing around with it and see if it can work.
From Inform the System About Lengthy User-Initiated Activities:
Denoting user-initiated work prevents the system from deferring the operations or putting your app in App Nap
If we are sure App Nap is causing issues we can add code to call NSProcessInfo.beginActivityWithOptions passing NSActivityUserInitiated. Something like that.
I can enter an issue for this, but it would be nice to first confirm that App Nap is causing problems.
Testing seemed to show that macOS is using heristics to decide if App Nap should be prevented or not. Playing a small movie that can use hardware decoding shows napping being prevented some of the time:
But playing a 4K 60fps AV1 encoded video does not prevent App Nap:
I tried quick test by adding this code to AppDelegate.applicationDidFinishLaunching:
ProcessInfo.processInfo.beginActivity(options: [.userInitiated], reason: "IINA is playing video")
And Xcode did not show App Nap being prevented.
I also tried userInteractive, which was added for macOS 13. Again, no effect on App Nap according to Xcode.
I stopped playback and minimized IINA to the dock. After a while Activity Monitor showed Yes for App Nap.
I even tried [.idleSystemSleepDisabled, .latencyCritical, .userInteractive]. App Nap was not prevented.
Possibly a lot of the documentation is out of date and Apple decided macOS "knows better" and apps are no longer able to disable App Nap?
It looks like you need to save the variable it returns somewhere. Don't let it get garbage collected.
I tried this and it seemed to work:
var activity: NSObjectProtocol?
func applicationDidFinishLaunching(_ aNotification: Notification) {
Logger.log("App launched")
activity = ProcessInfo().beginActivity(options: ProcessInfo.ActivityOptions.userInitiated, reason: "Playing video, allegedly")
The only way I've been able to reliably activate App Nap was to pause my video and use ⌘H to hide the window, then wait a couple of minutes. It didn't always activate if I only occluded it. And only tried one case of playing a video in the background, which didn't activate. So I used IINA > Hide IINA and waited, which seemed definitive as a test.
The graph below is from one of my tests. Strange that "App Nap Prevention" was actually turned off while playing the video. Though I did play while fully occluded for several minutes and App Nap did not turn on.
So it looks promising to begin an "activity" like this when entering the play state and ending it when pausing/stopping. My only concern is about "Idle Prevention". I'm unfamiliar with this. Might it interfere with machine sleep?
The Energy report makes no sense to me.
The row label is App Nap Prevention. But underneath there is a Napping label, but it fails to show an associated color? Under that is a section that seems to be intended to describe the rows except two of the labels do not match up, App Nap, missing "Prevention" and the words are out of order in High CPU Utilization. And the order of the rows of descriptions do not match the table, as High CPU Utilization should be first. Very sloppy design..
Looking at the pictures here. It seems the Napping label is supposed to have a green square in front of it. I do see the green bar when Activity Monitor reports IINA is napping.
IINA does have a SleepPreventor, but that does not seem to be what Idle Prevention is reporting. I think this is reflecting IINA's use of timers?
The Energy report makes no sense to me.
Ahh, good observations. Yeah, it looks like it was thrown together and then forgotten.
Looking at the pictures here.
This is a lot more helpful. The green square disappearing is a red flag (pardon the clashing color references). And when I compare their graphs to what I actually see in Xcode when the app is in the App Nap state:
To me it looks like the App Nap Prevention and possibly Idle Prevention indicators are broken and shouldn't be relied upon. It makes no sense for App Nap and App Nap Prevention to both be active at the same time.
Moreover, this suggests that there is a solution in ProcessInfo:
This page looks useful. It looks like a "power assertion" is a synonym for the "activity" started frombeginActivityWithOptions, though I could be wrong.
I think it's reasonable to assume Xcode is broken. But there are other tools. The Preventing Sleep column in Activity Monitor could be useful, along with pmset -g assertions. But manual testing is always a reliable fallback...
The Energy report in that screenshot sure looks defective to me. What Xcode version? I'm using Xcode 15.2.
I'm thinking some of this information in Apple's documentation archive is out of date. IINA is using OpenGL, so according to that documentation it shouldn't be a candidate for App Nap.
Power assertions are separate from ProcessInfo assertions. IINA's SleepPreventor class calls IOPMAssertionCreateWithName to establish a power assertion that prevents the display from sleeping. Issue #4354 shows pmset -g assertions being used to investigate a problem involving power assertions.
There is also an App Nap column in Activity Monitor. I see that turning to Yes when I stop playback and minimize IINA to the dock.
What Xcode version? I'm using Xcode 15.2.
I'm using 15.3.
I'm thinking some of this information in Apple's documentation archive is out of date. IINA is using OpenGL, so according to that documentation it shouldn't be a candidate for App Nap.
In my experience, if a doc falls out of date Apple usually just deletes it. Notice they start with the word "Generally". It weakens the usefulness of the bullet points considerably. Probably they are changing the heuristics slightly with each release. You mentioned you saw App Nap engage with a video playing in the background. I wonder if it was muted or had very little audio output?
Power assertions are separate from ProcessInfo assertions. IINA's SleepPreventor class calls IOPMAssertionCreateWithName to establish a power assertion that prevents the display from sleeping. Issue https://github.com/iina/iina/issues/4354 shows pmset -g assertions being used to investigate a problem involving power assertions.
Ahh, thank you. And I see no mention of App Nap in IOKit.pwr_mgt. Conway's Law in action. So there's probably not much useful in Power Management for this problem.
My best bet remains to do testing of processInfo.beginActivity to see if it is strong enough to prevent the app from napping but weak enough not to prevent the machine from sleeping.
@svobs
but adding or removing the titled style seems to be a special case which has many side effects.
Is removing titled strictly necessary to achieve the "pseudo full screen" effect? Setting fullSizeContentView + titlebarAppearsTransparent + then manually hiding/showing the titlebar as needed (via setting opacity?) seems like it might be able to achieve the same.
Anyway, are we sure that it's actually AppNap causing issues here? The suspending of GL draws may well be something CAOpenGLLayer specific and not necessarily app nap related.