Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarking with asv #1761

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Benchmarking with asv #1761

wants to merge 7 commits into from

Conversation

VeckoTheGecko
Copy link
Contributor

@VeckoTheGecko VeckoTheGecko commented Nov 13, 2024

This PR introduces benchmarking infrastructure to the project via asv. Benchmarks can be run on a pull request by adding the run-benchmarks label to it. Two environments will be created in the CI runner with the prior and proposed changes, and both suites of benchmarks will be run and compared against each other.

Note that this PR only has example benchmarks for the timebeing until we can discuss benchmarks of interest.

The running of the benchmarks in CI is only one aspect of the benchmarking (ie, only for core parcels functionality). Using asv, we can create different suites of benchmarks (e.g., one for CI, and one for more heavy simulations). The benefit of using asv is everything else that comes out of the box with it, some being:

  • being able to run benchmarks easily across several commits, visualising them in a dashboard
  • easily create and manage these benchmarking environments
  • profiling support to dive into locations of slowdowns
  • community support

Changes:

  • asv configuration (conf file, benchmarks folder, and CI workflow)
  • asv documentation (available via the community page in the maintainer section)

I have done some testing of the PR label workflow in VeckoTheGecko#10 . We can only test this for PRs in OceanParcels/parcels when its in master


Related to #1712

@VeckoTheGecko
Copy link
Contributor Author

@erikvansebille On the topic of performance, are you also experiencing it taking something like 10s occasionally to run import parcels?

@erikvansebille
Copy link
Member

@erikvansebille On the topic of performance, are you also experiencing it taking something like 10s occasionally to run import parcels?

Yes, I also experience this slow import sometimes. Not sure why...

@VeckoTheGecko
Copy link
Contributor Author

Setting to draft until we have some actual benchmarks that we can include in this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

Successfully merging this pull request may close these issues.

2 participants