Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test suite clean up #3385

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open

Test suite clean up #3385

wants to merge 87 commits into from

Conversation

JDBetteridge
Copy link
Member

@JDBetteridge JDBetteridge commented Feb 2, 2024 •
edited by connorjward
Loading

Description

  • Switches to having MPI executed 'on the outside'. This means we are no longer bound to MPICH (the source of huge build times and MacOS bugs) and parallel tests do not oversubscribe the runners.
  • Includes various associated bug fixes since parallel tests are not run in isolation.

TODOs

  • Fix remaining failing tests
  • Add make check and clean up Makefile (and use in CI + mention on install page)
  • Remove --download-mpich from everywhere, means that MPI is a system dependency (do as part of switch to pip)

@connorjward
Copy link
Contributor

connorjward commented Feb 2, 2024 •
edited
Loading

This is cool, but isn't it a bad idea to effectively remove test coverage? If CI doesn't run all the tests no one will.

I can see this being useful in the context of a bigger change where we run the test suite with a number of Firedrake configurations and only one of them would run these slow tests.

@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from ecb9628 to f3685b7 Compare February 9, 2024 19:21
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from f3685b7 to f848c6f Compare March 5, 2024 13:40
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch 2 times, most recently from 93a0aae to a20fc9d Compare June 7, 2024 13:52
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from a5f614f to 27b9af3 Compare July 19, 2024 16:25
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from 2a7f468 to 6ed1774 Compare August 18, 2024 13:44
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch 2 times, most recently from 8132b64 to 1122a3b Compare August 31, 2024 18:37
@JDBetteridge JDBetteridge self-assigned this Sep 4, 2024
JHopeCollins
JHopeCollins previously requested changes Sep 10, 2024
View reviewed changes
tests/regression/test_ensembleparallelism.py Outdated Show resolved Hide resolved
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch 2 times, most recently from 9d5f056 to df4aea3 Compare September 12, 2024 13:04
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 12, 2024 •
edited
Loading

TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake complex8118 ran6553 passed1565 skipped0 failed

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 12, 2024 •
edited
Loading

TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake real7989 ran7280 passed709 skipped0 failed

@JDBetteridge JDBetteridge marked this pull request as ready for review September 24, 2024 14:23
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from bf317f5 to ef87021 Compare October 2, 2024 21:57
@connorjward connorjward changed the title Mark and skip slow tests Test suite clean up Oct 3, 2024
connorjward
connorjward reviewed Oct 3, 2024
View reviewed changes
Copy link
Contributor

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally very happy with this.

.github/workflows/build.yml Outdated Show resolved Hide resolved
.test_durations Outdated Show resolved Hide resolved
firedrake/parameters.py Outdated Show resolved Hide resolved
firedrake/slate/slac/compiler.py Show resolved Hide resolved
firedrake/tsfc_interface.py Outdated Show resolved Hide resolved
firedrake/tsfc_interface.py Outdated Show resolved Hide resolved
tests/demos/test_demos_run.py Outdated Show resolved Hide resolved
tests/output/test_io_mesh.py Outdated Show resolved Hide resolved
tests/slate/test_hdg_poisson.py Outdated Show resolved Hide resolved
@JDBetteridge JDBetteridge force-pushed the JDBetteridge/faster_tests branch from 8435a69 to 10f7f0c Compare October 8, 2024 14:02
@connorjward connorjward dismissed stale reviews from JHopeCollins and themself January 10, 2025 17:28

I'm in charge now.

@connorjward connorjward mentioned this pull request Jan 21, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@connorjward connorjward connorjward left review comments

@JHopeCollins JHopeCollins JHopeCollins left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@connorjward connorjward

Labels
bug enhancement macOS performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

INSTALL: Tests not passing on fresh install M2 Mac
4 participants
@JDBetteridge @connorjward @JHopeCollins @Ig-dolci