Performance optimizations to reduce large repository loading time
This pull request optimizes MunkiAdmin's performance when loading large Munki repositories by eliminating Core Data query bottlenecks and expensive object creation operations.
With my organization's repository containing 1000+ packages, these changes reduced loading time from 10.0 seconds to 2.1 seconds. (On macOS 26.0, running Xcode 26.0.1.)
I had some help solving this with an AI assistant, which helped break down the initial load time observations as follows:
| Operation | Time (ms) | Percentage |
|---|---|---|
| Catalog Processing | 2,287 | 26.9% |
| Icon Processing | 2,070 | 24.3% |
| Developer Processing | 1,787 | 21.0% |
| Category Processing | 1,286 | 15.1% |
| Total Bottlenecks | 7,430 | 87.3% |
The solution ended up being four similar improvements involving dictionary caching.
1. Catalog Dictionary Caching
- Problem: ~3,432 Core Data queries executed for catalog lookups
- Solution: Pre-populate lookup dictionary, replace queries with O(1) lookups
- Result: 99.8% improvement (2,287ms → 5ms)
2. IconImageMO Object Caching
- Problem: ~1,144 expensive
createIconImageFromURLcalls with file I/O and image processing - Solution: Pre-cache all IconImageMO objects during initialization
- Result: 99.6% improvement (2,070ms → 9ms)
3. Developer Dictionary Caching
- Problem: ~1,144 Core Data queries for developer object lookups
- Solution: Pre-populate lookup dictionary, replace queries with O(1) lookups
- Result: 99.5% improvement (1,787ms → 8ms)
4. Category Dictionary Caching
- Problem: ~1,144 Core Data queries for category object lookups
- Solution: Pre-populate lookup dictionary, replace queries with O(1) lookups
- Result: 99.5% improvement (1,286ms → 7ms)
After these changes were made, I observed a significant improvement:
| Metric | Before | After | Improvement |
|---|---|---|---|
| Package Processing | 8,645ms | 1,427ms | -83.5% (-7,218ms) |
| Total Loading Time | ~10.0s | ~2.1s | -79% (-7.9s) |
| Core Data Queries | ~5,725 | ~29 | -99.5% (-5,696) |
These changes contributed to an overall 79% improvement in loading time across multiple test runs, including both initial load and reloading (using Command-R). The difference was very noticeable.
No significant changes in memory usage was observed after these changes. The changes should follow existing code patterns, and should maintain full backward compatibility with all macOS versions that run MunkiAdmin.
I'm hopeful this may have similar benefits for others with large Munki repos. Thanks for considering!
Thank you for this, looks interesting and the performance improvements are significant! I wanted to get the current release out before taking a closer look at this. Will get back to you soon.