chore: merge develop to feature/test-burn-in (do not squash) #28169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Simon Törnqvist <[email protected]>
…7770) * basic card visuals, working on click * visual tweaks and i18n * match other i18n * redo duration format, should make common too * basic debug, needs validation of relevant run * refactor and move files * working top level state * working grouping, needs subcomponent * switch click target * remove dayjs change * fix current commit * revise commit for message * working view runs button * fix type check * fix tests * fix integration tests * remove old test * add layout * add component test for new layout and cleanup * add count check * standardize cy tags * revise count test * fix typos * fix padding * start wrapping * happy wrap, next trunc * responsive if wonky * clean up and titles * magic resize, needs tests per breakpoint * standardize the rollup * working default rollup * rollup spacing tweaks * center elements * update tests for multiple viewports * add missing breakpoint tests * fix debug tests for status and badge * remove unneeded check * add functional component * fix check_ts * update container test * add changelog * add tooltip, needs aria label and external link update * fix overflow * fix avatar and add aria label to count summary * change to external href * remove popup sizing * add tests * add missing data cy * fix runcard test * Truncate result count * make truncate style conditional * match design truncate position * adddress PR comments * chore:updating styles (#27798) * add tooltip content tests and skeletons * clean up tooltip validation * fix runs selector * Update packages/app/src/runs/RunsSkeletonRow.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunTagCount.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunsSkeleton.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunTagCount.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunsSkeleton.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunsSkeletonRow.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunTagCount.vue Co-authored-by: Stokes Player <[email protected]> * Update packages/app/src/runs/RunTagCount.vue Co-authored-by: Stokes Player <[email protected]> * remove redundant shrink-1s * more shrinks --------- Co-authored-by: Stokes Player <[email protected]>
…27821) * chore: update chrome PR creation workflow to add to firewatch board * chore: adding in ability to manually run job * chore: changing how we add the chrome update PR to the firewatch board
…#27778) * chore: update kpi report generation to include feature review metrics * chore: fix typo
…27837) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Ben M <[email protected]>
#27874) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Ben M <[email protected]>
* docs: update installation requirements in contributing.md * docs: update installation requirements in contributing.md * docs: update installation requirements in contributing.md Co-authored-by: Mike McCready <[email protected]> --------- Co-authored-by: Mike McCready <[email protected]>
https://lodash.com/docs/#inRange `end` is exclusive, so it needs to be set to 1000 to allow status code 999 to be included (as per documented). Co-authored-by: Adam Stone-Lord <[email protected]> Co-authored-by: Ben M <[email protected]>
Co-authored-by: Matt Schile <[email protected]> Co-authored-by: Matt Schile <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>
…#28132) Co-authored-by: Cacie Prins <[email protected]>
* chore: set up feature/test-burn-in feature branch * feat: add burnIn Configuration option (currently a no-op) (#27377) * feat: add the burnIn Configuration to the config package. Option currently is a no-op * chore: make burn in experimental * chore: set experimentalBurnIn to false by default * feat: add new experimental retries configuration (#27412) * feat: implement the experimental retries configuration options to pair with test burn in * [run ci] * fix cache invalidation [run ci] * fix snapshot added in v13 for module api to include test burn in experimentalflag * chore: fix merge conflict * chore: add burnInTestAction capability (#27768) * add burnInTestAction capability * feat: add burn in capability for cloud * chore: fix snapshot for record_spec * feat: implement experimental retries (#27826) * chore: format the retries/runner snapshot files to make diff easier * feat: implement experimentalRetries strategies 'detect-flake-and-pass-on-threshold' and 'detect-flake-but-always-fail'. This should not be a breaking change, though it does modify mocha and the test object even when the experiment is not configured. This is to exercise the system and make sure things still work as expected even when we go GA. Test updates will follow in following commits. * chore: update snapshots from system tests and cy-in-cy tests that now have the cypress test metadata property _cypressTestStatusInfo. tests have been added in the fail-with-[before|after]each specs to visually see the suite being skipped when developing. * chore: add cy-in-cy tests to verify reporter behavior for pass/fail tests, as well as new mocha snapshots to verify attempts. New tests were needed for this as the 'retries' option in testConfigOverrides currently is and will be invalid for experiment and will function as an override. tests run in the cy-in-cy tests are using globally configured experimentalRetries for the given tested project, which showcases the different behavior between attempts/retries and pass/fail status. * chore: add unit test like driver test to verify the test object in mocha is decorated/handled properly in calculateTestStatus * chore: add sanity system tests to verify console reporter output for experimental retries logic. Currently there is a bug in the reporter where the logged status doesnt wait for the aftereach to complete, which impacts the total exitCode and printed status. * fix: aftereach console output. make sure to fail the test in the appropriate spot in runner.ts and not prematurely, which in turn updates the snapshots for cy-in-cy as the fail event comes later." * chore: address comments from code review * fix: make sure hook failures print outer status + attempts when the error is the hook itself. * chore: improve types within calculateTestStatus inside mocha.ts * Revert "feat: add burnIn Configuration option (currently a no-op) (#27377)" This reverts commit c428443. * Revert "chore: add burnInTestAction capability (#27768)" This reverts commit ae3df1a. * chore: run snapshot and binary jobs against experimental retries feature branch * chore: add changelog entry (wip) * Revert "fix snapshot added in v13 for module api to include test burn in experimentalflag" This reverts commit bb5046c. * Fix system tests * Clear CircleCI cache * Normalize retries config for test execution * Fixed some unit tests * update snapshots for newer test metadata * Fix cy-in-cy snapshots * update snapshots * bump cache version * chore: ensure legacy retry overrides work; reject exp. retries overrides (#28045) * update changelog * flip if statement in experimental retries option validation * refactor invalid experimental retry override for more useful error msg * revert testConfigOverrides snapshot * update snapshots for test override sys test * Update packages/config/src/validation.ts Co-authored-by: Chris Breiding <[email protected]> * succinct changelog entry; links to docs for details * testConfigOverride system test snapshots * Update .github/workflows/update_v8_snapshot_cache.yml Co-authored-by: Ryan Manuel <[email protected]> * Update cli/CHANGELOG.md Co-authored-by: Ryan Manuel <[email protected]> * Update packages/driver/src/cypress.ts Co-authored-by: Ryan Manuel <[email protected]> * updating cache-version * improve typescript usage when appending experimental retry options to experiments in Experimenets.vue * Revert "improve typescript usage when appending experimental retry options to experiments in Experimenets.vue" This reverts commit b459aba. * refactor test config override validation for experimental retry subkeys * account for error throw differences in browsers in system tests * bump circle cache * bump circle cache again --------- Co-authored-by: astone123 <[email protected]> Co-authored-by: mabela416 <[email protected]> Co-authored-by: Muaz Othman <[email protected]> Co-authored-by: Muaz Othman <[email protected]> Co-authored-by: Cacie Prins <[email protected]> Co-authored-by: Cacie Prins <[email protected]> Co-authored-by: Chris Breiding <[email protected]> Co-authored-by: Ryan Manuel <[email protected]> Co-authored-by: Matthew Schile <[email protected]>
Co-authored-by: Matt Schile <[email protected]>
cacieprins
changed the title
ryanthemanuel
approved these changes
Oct 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Additional details
This gets the test burn in branch up to date with develop. Do not squash - we want to preserve this git history so the feature branch can merge nicely into develop.
Steps to test
How has the user experience changed?
PR Tasks
cypress-documentation?type definitions?