Xcode tools to measure Metal Performance

[roblabs]

Xcode has a set of tools with excellent documentation on how to Optimize Performance with GPU Counters for measuring performance of our Metal based engine.

Opening this issue for any Game Developers or teams that want to aid in measuring the performance of the MapLibre solution.

Items to review

• [ ] Using Metal System Trace in Instruments to Profile Your App - Using the Game Performance Profile in Xcode Instruments
• [ ] Optimizing Performance with GPU Counters
• [ ] Enabling Frame Capture
• [ ] WWDC Videos
  • Optimize Metal apps and games with GPU Counters
  • Explore Live GPU Profiling with Metal Counters

---

Measuring FPS on a device with the MapLibre Swift UI app iosapp-swiftui

[roblabs]

Results from an Instruments profile from an iPhone SE using the Game Performance Profile.

An Average Frame Time duration of 16.67 ms is one 60 fps frame.

[roblabs]

Enabling the Frame Capture option in the Run Scheme also allows for more insight. And if nothing else some "eye candy" on how a map is drawn.

---

GPU Trace of MapLibre by enabling the Metal GPU Counters

https://user-images.githubusercontent.com/118112/115421507-dda7a680-a1b0-11eb-8a1f-0759adcfd410.mov

[juhieta]

Very nice and useful! 👍

[roblabs]

As a test, I have adding some logging to the MapLibre Objective-C event handlers which allows us to measure the time difference between events (mapViewDidFinishLoadingStyle, mapViewDidFinishLoadingMap, mapViewDidBecomeIdle).

We are seeing multiple Metal Compiler Warning in the Xcode logs while debugging, and now this logging allows us to measure how much time is occurring between significant events.

There is also evidence that two of the events are firing twice. Timing-wise, it's not significant, but I'm not sure why that is happening. Full, un-snipped data at the bottom.

13:42:22.981669,[maplibre] mapViewWillStartLoadingMap
13:42:22.992816,[maplibre] mapViewWillStartLoadingMap
<snip>
13:42:26.902628,[maplibre] mapViewDidBecomeIdle
13:42:29.971837,[maplibre] mapViewDidBecomeIdle

---

Time Measurement of MapLibre test app after a warm install. It's not clear if Metal Logging or MetalANGLE is the source of the performance hit.

---

13:18:34.447676-0700 MapLibre GL[3531:830726] [Metal Compiler Warning] Warning: Compilation succeeded with:

program_source:54:12: warning: unused variable 'flippedFragCoord'
    float4 flippedFragCoord;
           ^
13:18:34.674035-0700 MapLibre GL[3531:830726] [Metal Compiler Warning] Warning: Compilation succeeded with:

program_source:62:12: warning: unused variable 'flippedFragCoord'
    float4 flippedFragCoord;
           ^
13:18:34.953372-0700 MapLibre GL[3531:830726] [Metal Compiler Warning] Warning: Compilation succeeded with:

program_source:79:11: warning: unused variable '_uv_linesofar'
    float _uv_linesofar = (floor(in._ua_data.z / 4.0) + (in._ua_data.w * 64.0)) * 2.0;
          ^
13:18:35.008865-0700 MapLibre GL[3531:830726] [Metal Compiler Warning] Warning: Compilation succeeded with:

program_source:75:12: warning: unused variable 'flippedFragCoord'
    float4 flippedFragCoord;
           ^
13:18:35.669612-0700 MapLibre GL[3531:830726] [Metal Compiler Warning] Warning: Compilation succeeded with:

program_source:86:12: warning: unused variable 'flippedFragCoord'
    float4 flippedFragCoord;
           ^

Raw Time stamp Data

13:42:22.728863,Metal GPU Frame Capture Enabled
13:42:22.730925,Metal API Validation Enabled
13:42:22.922441,MGLMapView WARNING UIViewController.automaticallyAdjustsScrollViewInsets is deprecated use MGLMapView.automaticallyAdjustContentInset instead.
13:42:22.981669,[maplibre] mapViewWillStartLoadingMap
13:42:22.981782,[maplibre] onWillStartLoadingMap
13:42:22.992816,[maplibre] mapViewWillStartLoadingMap
13:42:22.992887,[maplibre] onWillStartLoadingMap
13:42:23.026326,[maplibre] mapViewWillStartRenderingMap
13:42:23.026553,[maplibre] onWillStartRenderingMap
13:42:23.380332,[maplibre] mapViewDidFinishLoadingStyle
13:42:23.380722,[maplibre] onDidFinishLoadingStyle
13:42:25.821380,[Metal Compiler Warning] Warning: Compilation succeeded with:
13:42:25.997340,[Metal Compiler Warning] Warning: Compilation succeeded with:
13:42:26.262554,[Metal Compiler Warning] Warning: Compilation succeeded with:
13:42:26.289862,[Metal Compiler Warning] Warning: Compilation succeeded with:
13:42:26.878745,[Metal Compiler Warning] Warning: Compilation succeeded with:
13:42:26.884729,[maplibre] onDidFinishRenderingMap
13:42:26.884913,[maplibre] mapViewDidFinishLoadingMap
13:42:26.884986,[maplibre] onDidFinishLoadingMap
13:42:26.902628,[maplibre] mapViewDidBecomeIdle
13:42:26.902863,[maplibre] onDidBecomeIdle
13:42:29.971837,[maplibre] mapViewDidBecomeIdle
13:42:29.972060,[maplibre] onDidBecomeIdle

---

potentially Related

• https://developer.apple.com/forums/thread/659856
• https://developer.apple.com/forums/thread/661774

---

MetalANGLE Research questions (FB9120382):

• [x] Is this project using SceneKit? This project is not using SceneKit. It is using MetalANGLE.
• [ ] How is the app compiling its shaders? Shader compiling is likely occurring at run time, rather than build time. The open source community for MapLibre will need to investigate that optimization.
• [ ] Is it setting any particular language version or is it using the default for the system (ie. the latest for the OS)?

---

After a session at WWDC 21, I was able to research other options. Details also at developer.apple.com/forums/thread/659856.

• Be sure to test outside of Xcode, as Xcode does Extra Metal Validation. Here are two entries in our log related at the start of the Metal portion of the code.
  • Metal GPU Frame Capture Enabled
  • Metal API Validation Enabled
• In the Simulator, the container path changes, possibly every time (TBD), which could cause the Metal code to be re-compiled
• While using instrumentation via Xcode Instruments, be sure to watch for the MetalCompilerService in Instruments and the Console logs.
• Verify if the Metal Compiler Warnings go away in the condition after a reboot (“ this is expected to happen”).