sentry-react-native icon indicating copy to clipboard operation
sentry-react-native copied to clipboard
verified publisher icon getsentry

chore(internal): Update e2e tests driver

Open krystofwoldrich opened this issue 2 years ago • 5 comments

:loudspeaker: Type of change

  • [ ] Bugfix
  • [ ] New feature
  • [x] Enhancement
  • [ ] Refactoring

:scroll: Description

The Appium v2 stable version is out, our e2e tests were still using the beta. I could not run the beta on my Mac anymore.

:bulb: Motivation and Context

:green_heart: How did you test it?

run test locally, ci

:pencil: Checklist

  • [x] I reviewed submitted code
  • [x] I added tests to verify changes
  • [x] No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • [x] All tests passing
  • [x] No breaking changes

:crystal_ball: Next steps

#skip-changelog

krystofwoldrich avatar Jan 08 '24 09:01 krystofwoldrich

Android (legacy) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 398.67 ms 428.74 ms 30.06 ms
Size 17.73 MiB 19.93 MiB 2.19 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
abb7058e207628f7d3e5ebee6c410c2cf6c361f4 370.27 ms 389.58 ms 19.31 ms
3ffcddd6248046202afca78f6b9af8e3f591202c 302.92 ms 315.80 ms 12.88 ms
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 430.68 ms 456.13 ms 25.44 ms
d361d3886b2303280797f653160c781700570edf 354.10 ms 381.69 ms 27.59 ms
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 326.93 ms 330.14 ms 3.21 ms
b1e87125f5dbad623a4e4293384e68e8f702cc19 462.11 ms 465.71 ms 3.60 ms
25343377820b65409b646b44bc8e9bc9fbb51c8c 394.15 ms 415.12 ms 20.97 ms
34aba0802bf802288f91fada55c0b133252fe67f 328.10 ms 342.84 ms 14.74 ms
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 335.72 ms 355.52 ms 19.80 ms
31fcca2b37834863733f6f21677efb4b7a490aa8 391.22 ms 414.78 ms 23.56 ms

App size

Revision Plain With Sentry Diff
abb7058e207628f7d3e5ebee6c410c2cf6c361f4 17.73 MiB 19.83 MiB 2.10 MiB
3ffcddd6248046202afca78f6b9af8e3f591202c 17.73 MiB 19.75 MiB 2.02 MiB
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 17.73 MiB 19.75 MiB 2.01 MiB
d361d3886b2303280797f653160c781700570edf 17.73 MiB 19.81 MiB 2.08 MiB
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 17.73 MiB 20.04 MiB 2.31 MiB
b1e87125f5dbad623a4e4293384e68e8f702cc19 17.73 MiB 19.75 MiB 2.02 MiB
25343377820b65409b646b44bc8e9bc9fbb51c8c 17.73 MiB 19.84 MiB 2.11 MiB
34aba0802bf802288f91fada55c0b133252fe67f 17.73 MiB 19.80 MiB 2.07 MiB
76d1bafa51ac0d20b127bc97b0f45201263636c3+dirty 17.73 MiB 20.04 MiB 2.31 MiB
31fcca2b37834863733f6f21677efb4b7a490aa8 17.73 MiB 19.90 MiB 2.17 MiB

github-actions[bot] avatar Jan 08 '24 09:01 github-actions[bot]

  • 0.73 e2e tests build are failing due to https://github.com/facebook/react-native/issues/42109#issuecomment-1880663873

krystofwoldrich avatar Jan 08 '24 09:01 krystofwoldrich

Android (new) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 360.30 ms 409.50 ms 49.20 ms
Size 7.15 MiB 8.11 MiB 988.68 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
d0bf49472b8e02ea40eab48e2e861b603450233e+dirty 253.73 ms 308.23 ms 54.49 ms
3853f4362ec819042552ffa9ed30e8f6627e60bf+dirty 278.12 ms 338.72 ms 60.60 ms
e73f4eddcecda26c0380795bce5ba97e1f11b91f+dirty 262.98 ms 311.02 ms 48.04 ms
9c48b2c9eb6422d8d51fcf56b1b4ab7841d2ec8a+dirty 270.82 ms 321.12 ms 50.30 ms
27ef4ee6faa6d5229b16fca7b728ca6b689ab916+dirty 296.71 ms 351.00 ms 54.29 ms
6e8584e8f1988b33cc24c85f32711c91358905fa+dirty 383.37 ms 400.84 ms 17.47 ms
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 259.04 ms 304.67 ms 45.63 ms
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 371.40 ms 377.70 ms 6.31 ms
e2b64fed057d4b01cf7ad79eb19eb3c8b8002a82+dirty 258.82 ms 304.26 ms 45.44 ms
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 335.40 ms 360.06 ms 24.67 ms

App size

Revision Plain With Sentry Diff
d0bf49472b8e02ea40eab48e2e861b603450233e+dirty 7.15 MiB 8.04 MiB 910.85 KiB
3853f4362ec819042552ffa9ed30e8f6627e60bf+dirty 7.15 MiB 8.08 MiB 959.34 KiB
e73f4eddcecda26c0380795bce5ba97e1f11b91f+dirty 7.15 MiB 8.09 MiB 965.94 KiB
9c48b2c9eb6422d8d51fcf56b1b4ab7841d2ec8a+dirty 7.15 MiB 8.07 MiB 947.16 KiB
27ef4ee6faa6d5229b16fca7b728ca6b689ab916+dirty 7.15 MiB 8.08 MiB 959.49 KiB
6e8584e8f1988b33cc24c85f32711c91358905fa+dirty 7.15 MiB 8.13 MiB 1002.18 KiB
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 7.15 MiB 8.03 MiB 903.20 KiB
8900e1acdc5e3dcbb40938d4ba5c2abad4cc86a4+dirty 7.15 MiB 8.03 MiB 901.79 KiB
e2b64fed057d4b01cf7ad79eb19eb3c8b8002a82+dirty 7.15 MiB 8.07 MiB 947.16 KiB
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 7.15 MiB 8.10 MiB 980.41 KiB

github-actions[bot] avatar Jan 09 '24 10:01 github-actions[bot]

iOS (new) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 1226.59 ms 1234.48 ms 7.89 ms
Size 2.92 MiB 3.48 MiB 575.52 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
80b2ce3d0ebe86bf1196944a2036912f6670295c+dirty 1245.12 ms 1262.04 ms 16.92 ms
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 1288.10 ms 1289.54 ms 1.44 ms
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 1276.90 ms 1280.92 ms 4.02 ms
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 1271.12 ms 1272.28 ms 1.16 ms
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 1280.06 ms 1285.26 ms 5.20 ms
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 1272.22 ms 1273.98 ms 1.76 ms
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 1276.40 ms 1279.14 ms 2.74 ms
457e29fc08e6edbebee646d095513d72728a5756+dirty 1256.71 ms 1258.50 ms 1.79 ms
12427f44d4b7746c431a369b538ec70ab2993c02+dirty 1224.90 ms 1231.40 ms 6.50 ms
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 1220.87 ms 1221.47 ms 0.60 ms

App size

Revision Plain With Sentry Diff
80b2ce3d0ebe86bf1196944a2036912f6670295c+dirty 2.92 MiB 3.40 MiB 492.75 KiB
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 2.92 MiB 3.40 MiB 488.06 KiB
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 2.92 MiB 3.43 MiB 524.50 KiB
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 2.92 MiB 3.39 MiB 487.34 KiB
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 2.92 MiB 3.44 MiB 533.26 KiB
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 2.92 MiB 3.40 MiB 494.39 KiB
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 2.92 MiB 3.37 MiB 464.32 KiB
457e29fc08e6edbebee646d095513d72728a5756+dirty 2.92 MiB 3.43 MiB 524.75 KiB
12427f44d4b7746c431a369b538ec70ab2993c02+dirty 2.92 MiB 3.44 MiB 533.29 KiB
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 2.92 MiB 3.43 MiB 529.76 KiB

github-actions[bot] avatar Jan 09 '24 11:01 github-actions[bot]

iOS (legacy) Performance metrics :rocket:

  Plain With Sentry Diff
Startup time 1227.19 ms 1233.45 ms 6.26 ms
Size 2.36 MiB 2.92 MiB 569.94 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
80b2ce3d0ebe86bf1196944a2036912f6670295c+dirty 1265.92 ms 1268.60 ms 2.69 ms
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 1252.38 ms 1275.04 ms 22.66 ms
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 1258.57 ms 1267.32 ms 8.75 ms
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 1264.38 ms 1290.06 ms 25.68 ms
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 1256.10 ms 1259.08 ms 2.98 ms
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 1244.47 ms 1264.14 ms 19.67 ms
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 1247.06 ms 1274.58 ms 27.52 ms
457e29fc08e6edbebee646d095513d72728a5756+dirty 1253.94 ms 1269.18 ms 15.24 ms
12427f44d4b7746c431a369b538ec70ab2993c02+dirty 1267.15 ms 1271.30 ms 4.15 ms
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 1225.08 ms 1230.26 ms 5.17 ms

App size

Revision Plain With Sentry Diff
80b2ce3d0ebe86bf1196944a2036912f6670295c+dirty 2.36 MiB 2.84 MiB 486.98 KiB
d7401ac44acf524fe4d9a6a525f2cc13c70ce9e5+dirty 2.36 MiB 2.83 MiB 481.14 KiB
e5c9b8b6d5ce209fa4b3a77aa9de65bd590fa727+dirty 2.36 MiB 2.87 MiB 520.43 KiB
acadc0f974a0c234a3d9010117b076792d14d6e8+dirty 2.36 MiB 2.83 MiB 480.37 KiB
728164bd341a3b0e14876d86101c4bcca5b1f1ed+dirty 2.36 MiB 2.88 MiB 530.38 KiB
3ffcddd6248046202afca78f6b9af8e3f591202c+dirty 2.36 MiB 2.84 MiB 489.60 KiB
9a3ca655542f9224aa21b4838b439748a3043b4b+dirty 2.36 MiB 2.82 MiB 462.89 KiB
457e29fc08e6edbebee646d095513d72728a5756+dirty 2.36 MiB 2.87 MiB 520.67 KiB
12427f44d4b7746c431a369b538ec70ab2993c02+dirty 2.36 MiB 2.88 MiB 530.38 KiB
25343377820b65409b646b44bc8e9bc9fbb51c8c+dirty 2.36 MiB 2.88 MiB 525.47 KiB

github-actions[bot] avatar Jan 09 '24 14:01 github-actions[bot]