-
-
Notifications
You must be signed in to change notification settings - Fork 337
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: e2e replay #3993
base: main
Are you sure you want to change the base?
test: e2e replay #3993
Conversation
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Krystof Woldrich <[email protected]>
Co-authored-by: Krystof Woldrich <[email protected]>
Android (legacy) Performance metrics 🚀
|
Revision | Plain | With Sentry | Diff |
---|---|---|---|
27ef4ee | 317.40 ms | 321.70 ms | 4.30 ms |
6e8584e | 447.10 ms | 474.71 ms | 27.61 ms |
148f924 | 492.65 ms | 500.28 ms | 7.63 ms |
34aba08 | 328.10 ms | 342.84 ms | 14.74 ms |
70e6261 | 482.65 ms | 495.70 ms | 13.05 ms |
575f9da | 415.26 ms | 422.98 ms | 7.72 ms |
1c65324 | 426.37 ms | 460.36 ms | 33.99 ms |
15c80ab+dirty | 336.27 ms | 350.58 ms | 14.31 ms |
1d86dd6 | 405.14 ms | 411.06 ms | 5.92 ms |
31fcca2 | 391.22 ms | 414.78 ms | 23.56 ms |
App size
Revision | Plain | With Sentry | Diff |
---|---|---|---|
27ef4ee | 17.73 MiB | 19.82 MiB | 2.08 MiB |
6e8584e | 17.73 MiB | 19.86 MiB | 2.12 MiB |
148f924 | 17.73 MiB | 19.94 MiB | 2.21 MiB |
34aba08 | 17.73 MiB | 19.80 MiB | 2.07 MiB |
70e6261 | 17.73 MiB | 19.94 MiB | 2.21 MiB |
575f9da | 17.73 MiB | 19.83 MiB | 2.10 MiB |
1c65324 | 17.73 MiB | 19.95 MiB | 2.21 MiB |
15c80ab+dirty | 17.73 MiB | 20.04 MiB | 2.31 MiB |
1d86dd6 | 17.73 MiB | 19.86 MiB | 2.12 MiB |
31fcca2 | 17.73 MiB | 19.90 MiB | 2.17 MiB |
iOS (legacy) Performance metrics 🚀
|
Revision | Plain | With Sentry | Diff |
---|---|---|---|
dadc233+dirty | 1223.20 ms | 1236.88 ms | 13.68 ms |
5571a20+dirty | 1203.57 ms | 1204.57 ms | 1.00 ms |
70e6261+dirty | 1220.09 ms | 1230.04 ms | 9.95 ms |
62a750b+dirty | 1216.60 ms | 1229.14 ms | 12.54 ms |
acadc0f+dirty | 1264.38 ms | 1290.06 ms | 25.68 ms |
6e8584e+dirty | 1274.50 ms | 1296.82 ms | 22.32 ms |
c398f67+dirty | 1219.67 ms | 1225.66 ms | 5.99 ms |
86d6d2c+dirty | 1267.55 ms | 1286.21 ms | 18.66 ms |
e5c9b8b+dirty | 1258.57 ms | 1267.32 ms | 8.75 ms |
31fcca2+dirty | 1209.17 ms | 1216.21 ms | 7.04 ms |
App size
Revision | Plain | With Sentry | Diff |
---|---|---|---|
dadc233+dirty | 2.36 MiB | 2.84 MiB | 486.85 KiB |
5571a20+dirty | 2.36 MiB | 2.92 MiB | 569.93 KiB |
70e6261+dirty | 2.36 MiB | 3.03 MiB | 680.42 KiB |
62a750b+dirty | 2.36 MiB | 2.92 MiB | 570.00 KiB |
acadc0f+dirty | 2.36 MiB | 2.83 MiB | 480.37 KiB |
6e8584e+dirty | 2.36 MiB | 2.88 MiB | 533.17 KiB |
c398f67+dirty | 2.36 MiB | 3.04 MiB | 696.27 KiB |
86d6d2c+dirty | 2.36 MiB | 2.82 MiB | 462.82 KiB |
e5c9b8b+dirty | 2.36 MiB | 2.87 MiB | 520.43 KiB |
31fcca2+dirty | 2.36 MiB | 2.90 MiB | 552.95 KiB |
iOS (new) Performance metrics 🚀
|
Revision | Plain | With Sentry | Diff |
---|---|---|---|
dadc233+dirty | 1266.52 ms | 1282.55 ms | 16.03 ms |
5571a20+dirty | 1228.09 ms | 1233.45 ms | 5.36 ms |
70e6261+dirty | 1224.90 ms | 1231.02 ms | 6.12 ms |
62a750b+dirty | 1228.12 ms | 1230.53 ms | 2.41 ms |
acadc0f+dirty | 1271.12 ms | 1272.28 ms | 1.16 ms |
6e8584e+dirty | 1271.71 ms | 1281.26 ms | 9.55 ms |
c398f67+dirty | 1227.31 ms | 1230.00 ms | 2.69 ms |
86d6d2c+dirty | 1291.62 ms | 1296.80 ms | 5.18 ms |
e5c9b8b+dirty | 1276.90 ms | 1280.92 ms | 4.02 ms |
31fcca2+dirty | 1222.04 ms | 1226.51 ms | 4.47 ms |
App size
Revision | Plain | With Sentry | Diff |
---|---|---|---|
dadc233+dirty | 2.92 MiB | 3.40 MiB | 492.53 KiB |
5571a20+dirty | 2.92 MiB | 3.48 MiB | 575.54 KiB |
70e6261+dirty | 2.92 MiB | 3.59 MiB | 686.11 KiB |
62a750b+dirty | 2.92 MiB | 3.48 MiB | 575.59 KiB |
acadc0f+dirty | 2.92 MiB | 3.39 MiB | 487.34 KiB |
6e8584e+dirty | 2.92 MiB | 3.44 MiB | 536.52 KiB |
c398f67+dirty | 2.92 MiB | 3.60 MiB | 701.89 KiB |
86d6d2c+dirty | 2.92 MiB | 3.37 MiB | 464.31 KiB |
e5c9b8b+dirty | 2.92 MiB | 3.43 MiB | 524.50 KiB |
31fcca2+dirty | 2.92 MiB | 3.46 MiB | 557.31 KiB |
Android (new) Performance metrics 🚀
|
Revision | Plain | With Sentry | Diff |
---|---|---|---|
52a8031+dirty | 330.72 ms | 358.76 ms | 28.03 ms |
86d6d2c+dirty | 267.21 ms | 325.24 ms | 58.04 ms |
2534337+dirty | 597.14 ms | 665.04 ms | 67.90 ms |
22e31b6+dirty | 295.75 ms | 346.73 ms | 50.98 ms |
dadc233+dirty | 363.19 ms | 370.37 ms | 7.18 ms |
148f924+dirty | 347.36 ms | 389.13 ms | 41.77 ms |
c398f67+dirty | 315.08 ms | 345.60 ms | 30.52 ms |
5bb8d5f+dirty | 356.71 ms | 389.65 ms | 32.94 ms |
d197b5c+dirty | 258.75 ms | 313.61 ms | 54.86 ms |
15c80ab+dirty | 276.38 ms | 327.54 ms | 51.17 ms |
App size
Revision | Plain | With Sentry | Diff |
---|---|---|---|
52a8031+dirty | 7.15 MiB | 8.09 MiB | 965.95 KiB |
86d6d2c+dirty | 7.15 MiB | 8.09 MiB | 962.69 KiB |
2534337+dirty | 7.15 MiB | 8.11 MiB | 988.68 KiB |
22e31b6+dirty | 7.15 MiB | 8.10 MiB | 981.29 KiB |
dadc233+dirty | 7.15 MiB | 8.04 MiB | 910.84 KiB |
148f924+dirty | 7.15 MiB | 8.21 MiB | 1.07 MiB |
c398f67+dirty | 7.15 MiB | 8.21 MiB | 1.07 MiB |
5bb8d5f+dirty | 7.15 MiB | 8.21 MiB | 1.06 MiB |
d197b5c+dirty | 7.15 MiB | 8.09 MiB | 962.72 KiB |
15c80ab+dirty | 7.15 MiB | 8.09 MiB | 966.13 KiB |
test/e2e/test/e2e.test.ts
Outdated
expect(replay).toBeDefined(); | ||
expect(replay.replay_id.length).toBe(32); | ||
|
||
const replayInfo = await fetchReplay(replay.replay_id); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought possibly the Android replays are uploaded due to a short time limit, but if I understand the code correctly, this fetch will wait up to 10 minutes, checking the endpoint every 1s.
Is that right? @vaind
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, nothing to do with timeouts. Replay capture doesn't work at all on android emulator in GHA at the moment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also no screenshots are captured by appium, if you look into an artifact for any of the Android jobs:
From what I can tell it's an old/recurring issue of the emulator on GHA, see actions/runner-images#6814
📢 Type of change
📜 Description
Tests that a replay is part of the error event when session replay options are enabled.
💡 Motivation and Context
💚 How did you test it?
📝 Checklist
sendDefaultPII
is enabled🔮 Next steps