`outdated --accurate*` takes excessively long (5+ minutes) or hangs when lots of MAS apps installed

[HugeIRL]

Configuration

mas ▁▁▁▁ 3.0.1
arch ▁▁▁ arm64
from ▁▁▁ homebrew/core/mas
origin ▁ https://github.com/mas-cli/mas.git
rev ▁▁▁▁ 67e5dfd18882ad294c58441f5d2308d4dfccb862
swift ▁▁ 6.2.1 (swiftlang-6.2.1.4.8 clang-1700.4.4.1)
driver ▁ 1.127.14.1
region ▁ CA
macos ▁▁ 26.1 (25B78)
mac ▁▁▁▁ Mac16,9
cpu ▁▁▁▁ Apple M4 Max

Bug description

I've noticed this on several machines. I wouldn't say I have a ton of MAS apps installed, less than 100. However mas outdated even with a pure 1Gbp/s pipe to my machine that is dedicated to that machine only can take longer then 5 minutes to complete. I have a script locally that exits the command on timeout of 5 minutes, which it always hits.

Updatest users are also reporting excessively long mas outdated runs on 3.0.0, which causes Updatest's internal timeout of 5 minutes to prevent command hanging to fail mas cli.

This did not happen on the old mas outdated command pre 3.0.0. I suspect the addition of a download/cancel to the logic is causing this issue.

Steps to reproduce

• Have an assortment of MAS apps (over 35)
• Run mas outdated
• In most cases, it takes over 5 minutes

[rgoldberg]

@HugeIRL outdated takes around 3 seconds for my 15-20 apps. I can't test this issue because I haven't gotten enough apps from the App Store.

Can you try moving all but 10 of your MAS apps out of /Applications (note that mas will also check any large apps folder you have configured, like /Volumes/<large app volume name>/Applications, so check your app count via mas list).

Then run time mas outdated. Can you continually add 10 apps back to /Applications, run time mas outdated & repeat? Then report back with the times?

If you notice a huge jump in time from N to N+10, could you zero in and find out exactly what count causes the massive jump?

I assume the problem is too many concurrent downloads. I can limit them to a maximum count. I'll make a WIP PR. The testing mentioned above will help me improve the maximum concurrent download count (which will either be hardcoded or processor-dependent for now as mas doesn't yet have a configuration system). If you can run that WIP when it's ready, and report if it helps, that would be very useful.

I was going to release 4.0.0 with the installd/PackageKit fix soon, but, I'd prefer to fix this issue & release 3.0.2 first, if possible, so time is of the essence.

Thanks.

[Accademia]

I’ve run into the same issue. I modified my own update script so that it can launch 16 parallel processes for mas outdated and 8 parallel processes for mas upgrade. After doing this, I’ve never had any timeouts again.

In reality, the App Store’s local background processes will block requests. After a certain number of queries, you must restart them; otherwise, even if you wait the whole night, the commands may never finish. I have nearly 300 MAS apps installed, and with high parallelism I’ve tested launching up to 128 concurrent queries — it can check all MAS apps within about one minute. However, after that, you must restart the App Store background processes, or subsequent MAS operations will stop responding completely (they’re basically stuck).

UpdateFull_For_MacOS

[HugeIRL]

@rgoldberg working atm so I can't really test it yet but I can later this evening.

Can you explain the reasoning behind adding the downloads at all to mas outdated? Just trying to understand the logic here so I can potentially fix this on my end and prevent you from having to do it through mas CLI. I don't believe the old mas outdated command ever tried to initiate a download?

[rgoldberg]

@Accademia Thanks for the limit numbers. That's exactly the type of solution I was planning. Please see my prior comment on this issue.

What CPU & Mac model are you using? The issue might be tied to core count, or it might be independent.

Have you tried larger values for the max concurrent downloads? I'd not, can you try & let me know what seems optimal as I code my WIP?

Thanks.

[Accademia]

I feel that the App Store’s local background processes will block requests. After a certain number of queries, you have to restart them; otherwise, even if you wait the whole night, the operations may never finish. I have nearly 300 MAS apps installed, and with high parallelism I’ve tested running up to 128 concurrent queries — it can check all MAS apps within about one minute. However, after that, you must restart the App Store background processes, or subsequent MAS commands become completely unresponsive (most likely because the processes are stuck).

I also tried increasing concurrency to 200 parallel processes, but macOS started throwing a large number of pop-up warnings saying that the system was not responding. After lowering the parallelism to 128, the problem disappeared entirely. My machine is an M3 Pro with 64GB of RAM. The relevant parallel code is shown below:

⸻

Parrale mas outdated （16 Process = -P 16）

mas list | awk '{print $1}' \
| xargs -n 1 -P 16 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec

⸻

Restart App Store agent process

# End the App Store agent process first (non-root)
killall appstoreagent 2>/dev/null

# Restart the installation daemon process (password required)
sudo killall installd

⸻

Full source code of the script

usercmd_updatefull

⸻

@Accademia Thanks for the limit numbers. That's exactly the type of solution I was planning. Please see my prior comment on this issue.

What CPU & Mac model are you using? The issue might be tied to core count, or it might be independent.

Have you tried larger values for the max concurrent downloads? I'd not, can you try & let me know what seems optimal as I code my WIP?

Thanks.

[rgoldberg]

@HugeIRL mas 3.1.0 will fix it, so no need to do anything except test. Thanks for offering.

There were numerous problems with the old outdated & upgrade that the 3.0.0+ download/cancel logic fixes:

1. The pre-3.0.0 logic compared versions from 2 different sources. The same release could have different version string from each, e.g., 1.8 & 1.0.0.8, or 10.0 & 26. I have no idea why Apple allowed that or developers do that, but it happens.
2. The information the pre-3.0.0 logic from Apple was only for the newest version of each app. That version might not be able to run on your macOS version, but there might be an older pending update that could. mas couldn't know about those.
3. Probably other issues, but the above 2 were bad enough.

The installd/PackageKit issue prompted me to release these changes as 3.0.0 sooner than I had anticipated, because I wanted people to be able to use them without the core changes to mas from the fix for the new issue from the forthcoming 4.0.0. I thus didn't have the chance to get feedback on 3.0.0.

[Accademia]

⸻

Parrale mas outdated （16 Process = -P 16）

mas list | awk '{print $1}' \
| xargs -n 1 -P 16 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec

The reason for forcibly terminating a process after 60 seconds on the command line is that some apps will pop up a dialog saying the app is no longer available on the App Store and require the user to click OK; without that click, the script hangs. To allow the update script to run unattended, we need to set a timeout. Under these conditions, each process can only check one app at a time to ensure that an app which hasn’t been checked yet isn’t killed along with the process about to be terminated.

[rgoldberg]

@Accademia Thanks for the info.

It sounds like you tried a max of 16 & a max of 128. Can you tell me how long each took?

If the higher amount took much less time, can you increase from 16 in increments (best to try lower amounts first to not get limited by Apple) to hone in on the optimal max?

Can you reply back with all of your run times for each max value?

Thanks.

[rgoldberg]

@Accademia For the aforementioned testing, can you temporarily move any apps that open dialogs out of your /Applications? Or just have your script skip them by name?

Thanks.

[rgoldberg]

@Accademia Are you saying that, even if you process the apps in 16-app chunks, you still must eventually kill your App Store processes? Or that the 16-app chunks avoid needing to kill the processes?

Thanks again.

[rgoldberg]

@HugeIRL @Accademia if the chunking still requires MAS processes to be restarted, then it might be best to offer both the old logic & the 3.x logic, so people can switch via a flag. I'm debating about which should be default. It really depends on what most users will experience, but, it's probably safest to make the default the old logic.

[HugeIRL]

@HugeIRL @Accademia if the chunking still requires MAS processes to be restarted, then it might be best to offer both the old logic & the 3.x logic, so people can switch via a flag. I'm debating about which should be default. It really depends on what most users will experience, but, it's probably safest to make the default the old logic.

I'd say old logic is safe as a default as it worked relatively fine for most people, and the new logic behind a flag of some sort.

[Accademia]

To avoid any misunderstanding, let me restate the requirements: • Parallelism = 16 processes. • Each process must handle exactly ONE app ID for the mas outdated check. It must not check multiple apps in a single process. The reason is: if an app triggers a pop-up saying it is no longer available in the App Store, that pop-up will freeze the process. If one process is checking multiple apps, that pop-up will block all remaining checks.

I do not want to manually maintain a list of app IDs that are no longer available on the App Store. Doing so would require me to constantly update the script myself. Instead, I want the script to run fully automatically, every night, for the entire year — completely unattended.

Therefore, all I need is: • mas outdated should detect which apps can be updated. • Then update only those apps. • Any apps that cannot be checked or updated should simply be skipped.

The above requirements are the functions that my current script has implemented.

https://github.com/Accademia/UpdateFull_For_MacOS

The effect: In MAS 3.x, the scanning(mas outdated) of 217 programs can be controlled to about 2 minutes. I'm satisfied with this. However, when calling mas upgrade one by one, mas will still report an error. I will further wait for the program to be fixed.

Regarding how different process counts affect execution time, see below:

Mas APP Total : 217

• 16 Process ： 2:47
• 64 Process ：2:19
• 128 Process ：When the number of processes reached 128, a large number of errors occurred. It may be because I have tried too many times before. Even if I restart the computer, Apple's servers no longer accept my query requests.

mas list | wc -l
     217

128 Process

 # Bash built-in 'time' formatting variables
TIMEFORMAT=$'Total elapsed time (wall clock): %R seconds\nUser CPU time: %U seconds\nSystem CPU time: %S seconds\nCPU utilization: %P'

time bash <<'SH'
mas list | awk '{print $1}' \
| xargs -n 1 -P 128 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec
'
SH
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}
Error: Error Domain=ISErrorDomain Code=3 "No bag entry" UserInfo={NSLocalizedDescription=No bag entry, NSLocalizedFailureReason=Could not find key 'buyProduct' in bag.}

64 Process

# Bash built-in 'time' formatting variables
TIMEFORMAT=$'Total elapsed time (wall clock): %R seconds\nUser CPU time: %U seconds\nSystem CPU time: %S seconds\nCPU utilization: %P'

time bash <<'SH'
mas list | awk '{print $1}' \
| xargs -n 1 -P 64 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec
'
SH
1037126344 Apple Configurator (2.18 -> 2.19)
Error: No downloads began
1136220934 Infuse (8.3.1 -> 8.3.2)
6746954963 Macrowave (1.1.3 -> 1.1.4)
1504416652 GeoGebra Calculator Suite (6.0.903 -> 6.0.906)
1607749385 Ghost Buster Pro (4.0.3 -> 4.0.4)
1444383602 Goodnotes (7.0.10 -> 7.0.14)
6504861501 Ghostery Privacy Ad Blocker (10.5.18 -> 10.5.19)
1482713641 moomoo (15.39.13808 -> 15.39.13818)
545519333 Prime Video (10.103 -> 10.106)
434808346 SimpleMind Pro (2.7.0 -> 2.8.0)
Error: No downloads began
858446756 World Clock Pro (1.8.6 -> 1.9.0)
bash <<<''  14.24s user 11.30s system 18% cpu 2:19.04 total

16 Process

#  Bash built-in 'time' formatting variables
TIMEFORMAT=$'Total elapsed time (wall clock): %R seconds\nUser CPU time: %U seconds\nSystem CPU time: %S seconds\nCPU utilization: %P'

time bash <<'SH'
mas list | awk '{print $1}' \
| xargs -n 1 -P 16 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec
'
SH
Error: No downloads began
1037126344 Apple Configurator (2.18 -> 2.19)
1504416652 GeoGebra Calculator Suite (6.0.903 -> 6.0.906)
1607749385 Ghost Buster Pro (4.0.3 -> 4.0.4)
1444383602 Goodnotes (7.0.10 -> 7.0.14)
6504861501 Ghostery Privacy Ad Blocker (10.5.18 -> 10.5.19)
1136220934 Infuse (8.3.1 -> 8.3.2)
6746954963 Macrowave (1.1.3 -> 1.1.4)
1482713641 moomoo (15.39.13808 -> 15.39.13818)
Error: No downloads began
545519333 Prime Video (10.103 -> 10.106)
434808346 SimpleMind Pro (2.7.0 -> 2.8.0)
858446756 World Clock Pro (1.8.6 -> 1.9.0)
bash <<<''  9.24s user 10.61s system 11% cpu 2:47.39 total

To ensure there are no blockages, you need to restart your computer before each test. Because it seems that the command I provided to kill the background process of the AppStore does not always have the effect. If you have a better way to quickly clear the blockage, you can also tell me.

# End App Store  Agent
killall appstoreagent 2>/dev/null
# Restart
sudo killall installd

Since I don’t know how to program, it takes ChatGPT Codex some time to generate code, so I might reply a bit slowly.

Thanks

@Accademia Thanks for the info.

It sounds like you tried a max of 16 & a max of 128. Can you tell me how long each took?

If the higher amount took much less time, can you increase from 16 in increments (best to try lower amounts first to not get limited by Apple) to hone in on the optimal max?

Can you reply back with all of your run times for each max value?

Thanks.

[Accademia]

@Accademia For the aforementioned testing, can you temporarily move any apps that open dialogs out of your /Applications? Or just have your script skip them by name?

Thanks.

For MAS apps that are currently running, my script checks whether the MAS app that is about to be updated is open. If it is, the script closes it first, performs the update, and then automatically relaunches it afterward.

However, it seems that in the newer versions of mas, the conflict stage has been moved from the installation phase to the new-version scanning phase. This indeed makes the situation more tricky.

My current workaround is: when the system popup appears, after waiting 60 seconds, the script kills the corresponding mas outdated process for that app. This prevents that one stuck process from blocking the remaining apps from being scanned.

The most reliable solution is actually to migrate more apps from the MAS version to the Homebrew version, because Homebrew updates do not require any application-state checks. As long as the proper NOPASSWD rules are configured in visudo, Homebrew updates can run completely silently — even if Excel is currently open. It can still be updated to the newer version without requiring Excel to be closed.

[rgoldberg]

I have a 16-download max concurrent outdated ready. I didn't have time to share a WIP PR. Will do so later tonight or tomorrow.

Will also reinstate the old logic, and gate the new logic for outdated & upgrade behind an --accurate flag. Maybe now or later I'll allow configuring the max concurrent download count for outdated.

[HugeIRL]

One thing I've experienced that I can only reproduce when I'm doing rapid updates for Updatest @rgoldberg is:

When I run mas outdated calls in rapid succession (maybe I'm rebuilding Updatest to test a fix), sometimes the App Store will pop up and ask me to sign in, even when I'm signed in, and error out if I hit cancel. Even after signing in, it'll pop up again.

I suspect rapid mas outdated calls on your new logic (like 15-20 within a few seconds) cause this and it's an edge case because I'm writing a UI that hooks into mas outdated, but wanted to highlight that.

Maybe I missed it, but what's the reason behind mas outdated needing to actually start/cancel a download? That seems to be the crux of a lot of these issues I'm having.

[rgoldberg]

@HugeIRL See https://github.com/mas-cli/mas/issues/1062#issuecomment-3553423361 for at least some of the reasons why the new logic is useful.

[HugeIRL]

@rgoldberg if it helps at all, I've had one of my Updatest users run into the problem as well:

Note here, they fail at the 1800 second time out for MAS CLI commands (30 minutes). They have roughly 134 installed applications with a big chunk of that being Mac App Store apps.

[HugeIRL]

Hey @rgoldberg some more info for you.

I've noticed that with the existing logic for outdated before your new fixes that launch soon, two problems:

1. mas outdated seems to take longer and longer the longer my machine is up. The actual time seems to only increase by about 2-3 seconds, but it's gotten to the point where mas outdated now takes 13 seconds on my system in the new logic, where as before it only took 2-3 seconds.

2. I had a user report (and several now) similar issues but that mas is straight up broken for them on 3.0.1

What's going on with me seems worse than what is being mentioned on GitHub. After maybe 2 apps being returned by mas outdated the whole thing just hangs. That's it. Just hangs forever. I'll run it again and just leave it to see if I ever get full results but functionally it's dead. 

I have no further information for you from that user (or the other reported users) other than a few of them are on Apple silicon and using MAS 3.0.1

[HugeIRL]

FWIW I think mas outdated should focus solely on getting you the list (or information for a single app id) if an app is outdated or not. I think the entire issues all my users experience here stems from the fact that it initiates a download. For a lot of users, especially those who may not have good internet connections, this kills MAS CLI for them completely.

[rgoldberg]

@HugeIRL Initiating then cancelling a download per app was solely to determine if an app is outdated or not (see https://github.com/mas-cli/mas/issues/1062#issuecomment-3553423361).

[HugeIRL]

@rgoldberg yes I know, just providing you as much information as I can.

[rgoldberg]

@HugeIRL the below made me think you thought there was another purpose to it.

FWIW I think mas outdated should focus solely on getting you the list (or information for a single app id) if an app is outdated or not.

I am almost ready with a PR to supply both logic versions, plus optional unknown app filtering for the new logic. I need to test it some more, and I need to finish one other unrelated improvement for 3.1.0, then will release. Then I need to rebase 4.0.0 on 3.1.0; to test that, then release it.

[HugeIRL]

All good! Probably just poor wording on my side.

[rgoldberg]

@HugeIRL Cool. I just wanted to ensure I answered any concerns you had.

[rgoldberg]

This issue has been mitigated in 3.1.0 by:

1. Capping the max concurrent outdated detection download count to 16. Maybe that could be user configurable in the future.
2. Changing the default outdated app detection logic to be the inaccurate pre-3.0.0 logic, while processing two new flags for outdated & update/upgrade for two variants of the new accurate logic, one of which mitigates one of the issues with the 3.0.0 logic at the expense of a few extra quickly network calls to Apple, but that also could ignore a few apps that could under very odd circumstances have updates, but that won't show up in that logic or the inaccurate logic.

I've left this issue open because more work can be done to optimize/configure the max concurrent count, and other changes could be made to improve the logic.

[HugeIRL]

if there's anything you want me to specifically test @rgoldberg let me know. I also have the entire Updatest user base who also reports issues regularly I can share if they relate to any changes.

[rgoldberg]

@HugeIRL thanks for the testing offer.

I imagine that it will be easiest to just release 4.0.0, and have you, your users, and everyone else test it. The commands are currently broken on macOS 26.1, etc., and my testing doesn't show any problems.

But if people want to test mas 4.0.0 built from HEAD after I merge 4.0.0 code into main in a few hours, but before I release 4.0.0, that won't hurt. I'll get back to you soon when 4.0.0 code is in main.

[HugeIRL]

Sure, just offering to help where I can!