test(profiling): regression test for #16545#16565
Conversation
Codeowners resolved as |
Performance SLOsComparing candidate taegyunkim/16545-regression-test (fb257d6) with baseline main (af51ae1) 📈 Performance Regressions (2 suites)📈 iastaspects - 117/117✅ add_aspectTime: ✅ 103.223µs (SLO: <130.000µs 📉 -20.6%) vs baseline: +0.8% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ add_inplace_aspectTime: ✅ 101.553µs (SLO: <130.000µs 📉 -21.9%) vs baseline: -0.6% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ add_inplace_noaspectTime: ✅ 28.217µs (SLO: <40.000µs 📉 -29.5%) vs baseline: -0.7% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ add_noaspectTime: ✅ 49.029µs (SLO: <70.000µs 📉 -30.0%) vs baseline: -0.2% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ bytearray_aspectTime: ✅ 252.183µs (SLO: <400.000µs 📉 -37.0%) vs baseline: +1.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ bytearray_extend_aspectTime: ✅ 630.507µs (SLO: <800.000µs 📉 -21.2%) vs baseline: -3.4% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.4% ✅ bytearray_extend_noaspectTime: ✅ 263.805µs (SLO: <400.000µs 📉 -34.0%) vs baseline: -2.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ bytearray_noaspectTime: ✅ 136.375µs (SLO: <300.000µs 📉 -54.5%) vs baseline: -4.7% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ bytes_aspectTime: ✅ 221.979µs (SLO: <300.000µs 📉 -26.0%) vs baseline: +1.9% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ bytes_noaspectTime: ✅ 134.251µs (SLO: <200.000µs 📉 -32.9%) vs baseline: +0.7% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.0% ✅ bytesio_aspectTime: ✅ 3.775ms (SLO: <5.000ms 📉 -24.5%) vs baseline: -0.7% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ bytesio_noaspectTime: ✅ 315.407µs (SLO: <420.000µs 📉 -24.9%) vs baseline: -0.8% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.6% ✅ capitalize_aspectTime: ✅ 89.014µs (SLO: <300.000µs 📉 -70.3%) vs baseline: +0.8% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ capitalize_noaspectTime: ✅ 251.369µs (SLO: <300.000µs 📉 -16.2%) vs baseline: -0.7% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.5% ✅ casefold_aspectTime: ✅ 89.014µs (SLO: <500.000µs 📉 -82.2%) vs baseline: +0.8% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ casefold_noaspectTime: ✅ 308.730µs (SLO: <500.000µs 📉 -38.3%) vs baseline: -1.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ decode_aspectTime: ✅ 86.826µs (SLO: <100.000µs 📉 -13.2%) vs baseline: +0.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ decode_noaspectTime: ✅ 153.660µs (SLO: <210.000µs 📉 -26.8%) vs baseline: -1.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ encode_aspectTime: ✅ 84.450µs (SLO: <200.000µs 📉 -57.8%) vs baseline: -0.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ encode_noaspectTime: ✅ 141.014µs (SLO: <200.000µs 📉 -29.5%) vs baseline: -1.5% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ format_aspectTime: ✅ 14.581ms (SLO: <19.200ms 📉 -24.1%) vs baseline: -0.3% Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +4.4% ✅ format_map_aspectTime: ✅ 16.384ms (SLO: <21.500ms 📉 -23.8%) vs baseline: -0.3% Memory: ✅ 43.155MB (SLO: <46.000MB -6.2%) vs baseline: +4.6% ✅ format_map_noaspectTime: ✅ 371.808µs (SLO: <500.000µs 📉 -25.6%) vs baseline: +0.3% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ format_noaspectTime: ✅ 301.601µs (SLO: <500.000µs 📉 -39.7%) vs baseline: -1.5% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ index_aspectTime: ✅ 124.038µs (SLO: <300.000µs 📉 -58.7%) vs baseline: +3.0% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ index_noaspectTime: ✅ 40.400µs (SLO: <300.000µs 📉 -86.5%) vs baseline: +0.1% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.7% ✅ join_aspectTime: ✅ 210.833µs (SLO: <300.000µs 📉 -29.7%) vs baseline: -1.6% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ join_noaspectTime: ✅ 142.642µs (SLO: <300.000µs 📉 -52.5%) vs baseline: ~same Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ ljust_aspectTime: ✅ 584.888µs (SLO: <700.000µs 📉 -16.4%) vs baseline: 📈 +15.4% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +4.3% ✅ ljust_noaspectTime: ✅ 258.857µs (SLO: <300.000µs 📉 -13.7%) vs baseline: -0.5% ✅ lower_aspectTime: ✅ 293.869µs (SLO: <500.000µs 📉 -41.2%) vs baseline: -1.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ lower_noaspectTime: ✅ 234.585µs (SLO: <300.000µs 📉 -21.8%) vs baseline: -1.1% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ lstrip_aspectTime: ✅ 0.272ms (SLO: <3.000ms 📉 -90.9%) vs baseline: -2.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ lstrip_noaspectTime: ✅ 0.177ms (SLO: <3.000ms 📉 -94.1%) vs baseline: +0.8% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.2% ✅ modulo_aspectTime: ✅ 14.255ms (SLO: <18.750ms 📉 -24.0%) vs baseline: -0.7% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +4.3% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.761ms (SLO: <19.350ms 📉 -23.7%) vs baseline: +0.2% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ modulo_aspect_for_bytesTime: ✅ 14.407ms (SLO: <18.900ms 📉 -23.8%) vs baseline: -0.7% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.5% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.656ms (SLO: <19.150ms 📉 -23.5%) vs baseline: +0.2% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.1% ✅ modulo_noaspectTime: ✅ 0.363ms (SLO: <3.000ms 📉 -87.9%) vs baseline: -0.5% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ replace_aspectTime: ✅ 18.530ms (SLO: <24.000ms 📉 -22.8%) vs baseline: +0.7% Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +4.4% ✅ replace_noaspectTime: ✅ 280.027µs (SLO: <300.000µs -6.7%) vs baseline: -1.0% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ repr_aspectTime: ✅ 312.083µs (SLO: <420.000µs 📉 -25.7%) vs baseline: -2.5% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ repr_noaspectTime: ✅ 46.571µs (SLO: <90.000µs 📉 -48.3%) vs baseline: -0.6% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ rstrip_aspectTime: ✅ 385.832µs (SLO: <500.000µs 📉 -22.8%) vs baseline: +1.4% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.4% ✅ rstrip_noaspectTime: ✅ 184.048µs (SLO: <300.000µs 📉 -38.7%) vs baseline: +0.9% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ slice_aspectTime: ✅ 183.163µs (SLO: <300.000µs 📉 -38.9%) vs baseline: +1.3% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ slice_noaspectTime: ✅ 54.930µs (SLO: <90.000µs 📉 -39.0%) vs baseline: +1.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ stringio_aspectTime: ✅ 4.394ms (SLO: <5.000ms 📉 -12.1%) vs baseline: 📈 +14.1% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.5% ✅ stringio_noaspectTime: ✅ 348.051µs (SLO: <500.000µs 📉 -30.4%) vs baseline: -2.2% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ strip_aspectTime: ✅ 267.649µs (SLO: <350.000µs 📉 -23.5%) vs baseline: -2.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ strip_noaspectTime: ✅ 176.673µs (SLO: <240.000µs 📉 -26.4%) vs baseline: -0.8% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ swapcase_aspectTime: ✅ 333.115µs (SLO: <500.000µs 📉 -33.4%) vs baseline: -1.4% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ swapcase_noaspectTime: ✅ 269.561µs (SLO: <400.000µs 📉 -32.6%) vs baseline: -1.8% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ title_aspectTime: ✅ 325.848µs (SLO: <500.000µs 📉 -34.8%) vs baseline: -0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ title_noaspectTime: ✅ 256.657µs (SLO: <400.000µs 📉 -35.8%) vs baseline: -3.2% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ translate_aspectTime: ✅ 492.496µs (SLO: <700.000µs 📉 -29.6%) vs baseline: -1.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ translate_noaspectTime: ✅ 427.555µs (SLO: <500.000µs 📉 -14.5%) vs baseline: -1.4% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ upper_aspectTime: ✅ 295.161µs (SLO: <500.000µs 📉 -41.0%) vs baseline: -1.7% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.5% ✅ upper_noaspectTime: ✅ 235.758µs (SLO: <400.000µs 📉 -41.1%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 516.495µs (SLO: <700.000µs 📉 -26.2%) vs baseline: 📈 +19.7% Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.4% ✅ ospathbasename_noaspectTime: ✅ 428.900µs (SLO: <700.000µs 📉 -38.7%) vs baseline: -1.2% Memory: ✅ 42.723MB (SLO: <46.000MB -7.1%) vs baseline: +4.6% ✅ ospathjoin_aspectTime: ✅ 633.571µs (SLO: <700.000µs -9.5%) vs baseline: +1.8% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +5.3% ✅ ospathjoin_noaspectTime: ✅ 635.124µs (SLO: <700.000µs -9.3%) vs baseline: +0.8% Memory: ✅ 42.487MB (SLO: <46.000MB -7.6%) vs baseline: +4.0% ✅ ospathnormcase_aspectTime: ✅ 355.427µs (SLO: <700.000µs 📉 -49.2%) vs baseline: +0.5% Memory: ✅ 42.526MB (SLO: <46.000MB -7.6%) vs baseline: +4.2% ✅ ospathnormcase_noaspectTime: ✅ 360.818µs (SLO: <700.000µs 📉 -48.5%) vs baseline: -0.2% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.5% ✅ ospathsplit_aspectTime: ✅ 494.322µs (SLO: <700.000µs 📉 -29.4%) vs baseline: +0.7% Memory: ✅ 42.821MB (SLO: <46.000MB -6.9%) vs baseline: +5.1% ✅ ospathsplit_noaspectTime: ✅ 499.774µs (SLO: <700.000µs 📉 -28.6%) vs baseline: -0.8% Memory: ✅ 42.585MB (SLO: <46.000MB -7.4%) vs baseline: +4.5% ✅ ospathsplitdrive_aspectTime: ✅ 378.994µs (SLO: <700.000µs 📉 -45.9%) vs baseline: +1.0% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +5.2% ✅ ospathsplitdrive_noaspectTime: ✅ 72.942µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.7% Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +4.5% ✅ ospathsplitext_aspectTime: ✅ 465.507µs (SLO: <700.000µs 📉 -33.5%) vs baseline: +0.4% Memory: ✅ 42.605MB (SLO: <46.000MB -7.4%) vs baseline: +4.6% ✅ ospathsplitext_noaspectTime: ✅ 468.826µs (SLO: <700.000µs 📉 -33.0%) vs baseline: -0.4% Memory: ✅ 42.684MB (SLO: <46.000MB -7.2%) vs baseline: +4.6% 🟡 Near SLO Breach (1 suite)🟡 tracer - 6/6✅ largeTime: ✅ 31.444ms (SLO: <32.950ms -4.6%) vs baseline: +0.4% Memory: ✅ 36.766MB (SLO: <39.250MB -6.3%) vs baseline: +4.7% ✅ mediumTime: ✅ 3.093ms (SLO: <3.200ms -3.3%) vs baseline: +0.2% Memory: ✅ 35.507MB (SLO: <38.750MB -8.4%) vs baseline: +5.6% ✅ smallTime: ✅ 364.856µs (SLO: <370.000µs 🟡 -1.4%) vs baseline: +4.9% Memory: ✅ 35.606MB (SLO: <38.750MB -8.1%) vs baseline: +5.4%
|
|
✨ Fix all issues with BitsAI or with Cursor
|
Description
Testing
Risks
Additional Notes