sentry-cocoa icon indicating copy to clipboard operation
sentry-cocoa copied to clipboard

fix: model/architecture reporting for profiling

Open armcknight opened this issue 3 years ago • 6 comments

Before this change, this is how profiles from these devices would report device models:

Device model
iPhone 13 mini iPhone14,4
macbook pro with M1 arm64
iMac w/ Intel CPU x86_64
iphone simulator on M1 mbp arm64
iphone simulator on intel imac x86_64

the issue is that mac apps don't report their model in the same way that the iphone implementation of this same API does, instead reporting the CPU architecture for macOS implementations.

This pr add reporting the architecture (as part of conforming to the new profile schema changes being done in https://github.com/getsentry/sentry-cocoa/pull/2203) and fixes the invocation of the necessary APIs to get the correct result for mac apps. Partly by changing from uname to sysctl, and also by using the correct options on calls to sysctl from iOS and macOS, as they are different (see https://www.cocoawithlove.com/blog/2016/03/08/swift-wrapper-for-sysctl.html#looking-for-the-source).

Here are the results after the change:

Device architecture model
iPhone 13 mini arm64e iPhone14,4
macbook pro with M1 arm64e MacBookPro18,3
iMac w/ Intel CPU x86_64 iMac20,1
iphone simulator on M1 mbp arm64e MacBookPro18,3
iphone simulator on intel imac x86_64 iMac20,1

So the macOS device model is reported the same as the iphone models were all along, and the architecture now being reported for the m1 mac is more precise as well: arm64e vs just the arm64 previously reported.

Questions

~1. Is it more correct for simulator architectures to be reported as i386 or x86_64? We can choose between which API to use depending on which result we want; both are possible.~ 2. Do we want the macOS device model name for simulator builds, or the device being simulated? We won't be able to get the CPU architecture for the simulated device; it'll always be the CPU of the macOS device. Maybe we should add another key for simulated_device_name; ~it's also not possible to get the hardware identifier (like iPhone14,4) but it is possible to get the human-readable name (like iPhone 13 mini) which is why I suggest putting it in a separate field.~ actually, I do get iPhone14,4 etc in this field now!

armcknight avatar Sep 21 '22 07:09 armcknight

Did you already look into how we get this data from SentryCrash? https://github.com/getsentry/sentry-cocoa/blob/ffd5fa8f60ef84ee6be0e35c2848cb83c6eb24d0/Sources/Sentry/SentryCrashIntegration.m#L187-L276

Answers

  1. Is it more correct for simulator architectures to be reported as i386 or x86_64? We can choose between which API to use depending on which result we want; both are possible.

I think x86_64 is more correct as it's the architecture of the mac running the simulator. With https://github.com/getsentry/sentry-cocoa/pull/2185, the simulators on M1 macs will show arm64. I don't understand why it should be i386.

  1. Do we want the macOS device model name for simulator builds, or the device being simulated? We won't be able to get the CPU architecture for the simulated device; it'll always be the CPU of the macOS device. Maybe we should add another key for simulated_device_name; it's also not possible to get the hardware identifier (like iPhone14,4) but it is possible to get the human-readable name (like iPhone 13 mini) which is why I suggest putting it in a separate field.

I think both would make sense here. An extra field like simulated_device_name would definitely make sense. It would be great if you could open up an issue or maybe a PR for this.

philipphofmann avatar Sep 21 '22 12:09 philipphofmann

@philipphofmann after clicking through the code paths it looks like you're talking about the spot I referenced here, at least w.r.t. getting the architecture? https://github.com/getsentry/sentry-cocoa/issues/900#issuecomment-1253295820

But I think it's going to be problematic in the same way for simulators running on macs. The reason I'm currently getting i386 is simply because CPU_TYPE_I386 is defined as CPU_TYPE_X86, and I just have the i386 macro first in the switch statement. But I don't think x86 is right either, I expected x86_64 🤔 CPU_TYPE_X86_64 is defined as CPU_TYPE_X86 | CPU_ARCH_ABI64 so maybe I have an additional check to make.

armcknight avatar Sep 21 '22 20:09 armcknight

Updates:

  • I ironed out the mac architecture returning x86_64 by modifying how the result of sysctlbyname is handled.

  • Also discovered that HW_MACHINE and HW_MODEL are deprecated, which are being used for some calls to sysctl, looking to see when they were first deprecated:

#define HW_MACHINE       1              /* string: machine class (deprecated: use HW_PRODUCT) */
#define HW_MODEL         2              /* string: specific machine model (deprecated: use HW_TARGET) */

Update: here is the deprecation: https://github.com/apple/darwin-xnu/commit/d4061fb0260b3ed486147341b72468f836ed6c8f#diff-c47401eadd378d5af4e9ed903849ac20c3de7156390e0b68b92adc5c963365e2L936-R944. xnu-7195.50.7.100.1 was apparently first released on 30 Oct 2020 according to theiphonewiki:

Darwin Kernel Version 20.1.0: Fri Oct 30 20:40:35 PDT 2020; root:xnu-7195.50.7~1/RELEASE_ARM64_T8010.

iOS 14 was released in 2020, so we could check for that to use the new recommended values.

  • I looked into just using TARGET_CPU_X86_64, TARGET_CPU_ARM64 etc but they can't be used to distinguish between arm64 and arm64e. There are also __is_target_arch(x86_64), __is_target_arch(arm64) and __is_target_arch(arm64e) from TargetConditionals.h that can do that, but I'm not sure which platforms/versions __is_target_arch is available on.

  • Added a test but the one snag there is that I can't get the unit test process to run on an iOS device, I'd need to create a UI test (and thus a test host app, or use one of the sample apps)

armcknight avatar Sep 22 '22 00:09 armcknight

Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 1231.33 ms 1255.46 ms 24.13 ms
Size 20.50 KiB 337.71 KiB 317.20 KiB

Baseline results on branch: master

Startup times

Revision Plain With Sentry Diff
be47c6c37e25c3c5d92cceb875c6f408f76017f8 1235.71 ms 1268.02 ms 32.31 ms
be47c6c37e25c3c5d92cceb875c6f408f76017f8 1216.35 ms 1237.80 ms 21.45 ms
0e2e5938cd3bf4c83cc9158efea42b5988395d09 1234.78 ms 1266.10 ms 31.32 ms
b6ef18d7fb768d643a5ad85ae741921678e325b4 1248.35 ms 1270.16 ms 21.82 ms
fbeb49aa4bf63d1ce6537cbc4d509564d1510617 1218.13 ms 1243.70 ms 25.57 ms
9231f608d4b6fe4ec24f878e3c669d45ab946721 1256.78 ms 1267.74 ms 10.96 ms
79779928f46de58cd3cdb19a4059cbc343e64204 1219.80 ms 1241.92 ms 22.12 ms
5025d2e2643a41b83ce05711508ca9f77caa8eb8 1245.14 ms 1268.58 ms 23.44 ms
e95889964a4ac54b552ca78c56109bd62c0ae21c 1233.02 ms 1261.86 ms 28.84 ms
e43ce746ccbe0b1431e57afa01812beb2fe8cead 1235.77 ms 1252.06 ms 16.29 ms

App size

Revision Plain With Sentry Diff
be47c6c37e25c3c5d92cceb875c6f408f76017f8 20.50 KiB 333.54 KiB 313.04 KiB
be47c6c37e25c3c5d92cceb875c6f408f76017f8 20.50 KiB 333.54 KiB 313.04 KiB
0e2e5938cd3bf4c83cc9158efea42b5988395d09 20.50 KiB 333.88 KiB 313.38 KiB
b6ef18d7fb768d643a5ad85ae741921678e325b4 20.51 KiB 332.90 KiB 312.40 KiB
fbeb49aa4bf63d1ce6537cbc4d509564d1510617 20.51 KiB 333.51 KiB 313.00 KiB
9231f608d4b6fe4ec24f878e3c669d45ab946721 20.51 KiB 333.58 KiB 313.07 KiB
79779928f46de58cd3cdb19a4059cbc343e64204 20.50 KiB 333.58 KiB 313.07 KiB
5025d2e2643a41b83ce05711508ca9f77caa8eb8 20.51 KiB 331.79 KiB 311.28 KiB
e95889964a4ac54b552ca78c56109bd62c0ae21c 20.51 KiB 331.92 KiB 311.41 KiB
e43ce746ccbe0b1431e57afa01812beb2fe8cead 20.51 KiB 335.49 KiB 314.99 KiB

Previous results on branch: armcknight/fix-model-reporting

Startup times

Revision Plain With Sentry Diff
9ddb68fd5046ce916007dc6b547b5ac81325b1f5 1232.10 ms 1248.70 ms 16.60 ms
f18ee273d8e8021aca03a1a06fa6c353857c8f79 1266.32 ms 1274.24 ms 7.92 ms
68e330249e1c69717ad6290f5106eeba41139718 1256.34 ms 1274.15 ms 17.80 ms
86d609c82a18c165a860da2ed87e80c40e604342 1224.96 ms 1250.94 ms 25.98 ms
5c6c2870d0e01827d3ccd59c40224ca5b6b09a93 1202.93 ms 1237.38 ms 34.45 ms
f5b5dad39741d12519a221812247f810466b599c 1240.00 ms 1261.82 ms 21.82 ms
71f8c91dfb7ada537c4db206c310dd97950a4494 1255.54 ms 1268.16 ms 12.62 ms
ce392aa60e0287b5d50adcf9b80ea4d406930297 1207.13 ms 1238.54 ms 31.41 ms
5be9a54fbabd0b69c54695f207c664b9cabcd464 1217.73 ms 1248.56 ms 30.83 ms
2758ed363e60f65d2b6f1c3ed61c458a043efed2 1213.12 ms 1250.15 ms 37.02 ms

App size

Revision Plain With Sentry Diff
9ddb68fd5046ce916007dc6b547b5ac81325b1f5 20.50 KiB 335.46 KiB 314.96 KiB
f18ee273d8e8021aca03a1a06fa6c353857c8f79 20.50 KiB 335.46 KiB 314.96 KiB
68e330249e1c69717ad6290f5106eeba41139718 20.50 KiB 335.46 KiB 314.96 KiB
86d609c82a18c165a860da2ed87e80c40e604342 20.50 KiB 335.46 KiB 314.96 KiB
5c6c2870d0e01827d3ccd59c40224ca5b6b09a93 20.50 KiB 335.46 KiB 314.96 KiB
f5b5dad39741d12519a221812247f810466b599c 20.51 KiB 332.48 KiB 311.97 KiB
71f8c91dfb7ada537c4db206c310dd97950a4494 20.50 KiB 335.39 KiB 314.89 KiB
ce392aa60e0287b5d50adcf9b80ea4d406930297 20.51 KiB 332.48 KiB 311.97 KiB
5be9a54fbabd0b69c54695f207c664b9cabcd464 20.50 KiB 335.38 KiB 314.88 KiB
2758ed363e60f65d2b6f1c3ed61c458a043efed2 20.51 KiB 332.48 KiB 311.98 KiB

github-actions[bot] avatar Sep 22 '22 23:09 github-actions[bot]

That’s a good idea @brustolin. Let me take care of it in s separate PR.

armcknight avatar Sep 23 '22 17:09 armcknight

I'm reworking the tests a bit, this is a tricky one.

armcknight avatar Sep 23 '22 23:09 armcknight

@armcknight, what's the state of this PR? Should I give you a full review?

philipphofmann avatar Sep 29 '22 13:09 philipphofmann

@philipphofmann this is still a draft

armcknight avatar Oct 04 '22 17:10 armcknight

Currently blocked on #2257... need to be able to use the TEST/TESTCI macros in some UI test targets in the sample projects to perform the correct asserts for a few things.

armcknight avatar Oct 04 '22 21:10 armcknight

Last thing needed here is to link watchkit into the home assistant integration test

armcknight avatar Oct 07 '22 13:10 armcknight