Skip to content

Migrate benchmarks#588

Open
HarryCampion wants to merge 8 commits intomainfrom
migrate-benchmarks
Open

Migrate benchmarks#588
HarryCampion wants to merge 8 commits intomainfrom
migrate-benchmarks

Conversation

@HarryCampion
Copy link
Collaborator

@HarryCampion HarryCampion commented Feb 7, 2026

This pull request reorganizes and modernizes the benchmarking and test utilities for the project. It moves benchmark analysis scripts and outputs to a unified location, updates Makefile targets to use uv run instead of direct Python calls, and eliminates the shared test_utils.py in favor of defining utilities directly in each test module. This improves maintainability, avoids import issues, and ensures all test scripts use up-to-date paths and conventions.

The most important changes include:

Benchmarking and Analysis Refactor:

  • Moved analyze_benchmarks.py from tests/test_rhiza/benchmarks/ to .rhiza/utils/, updated it to use _benchmarks as the output directory, and adjusted all paths accordingly. [1] [2]
  • Updated the Makefile (.rhiza/make.d/01-test.mk) so that benchmark outputs (JSON, HTML, histogram) are written to _benchmarks/ instead of inside the test directory. The analysis script is now run via uv run .rhiza/utils/analyze_benchmarks.py.
  • Removed the obsolete tests/test_rhiza/benchmarks/README.md and .gitignore files, as well as the tests/conftest.py and test_utils.py utility modules, consolidating benchmark documentation and utilities. [1] [2] [3] [4]

Makefile and Test Target Modernization:

  • Updated Makefile targets to use uv run pytest for running tests and generating coverage and reports, replacing direct calls to the Python interpreter in the virtual environment. Added checks for missing source or test folders, with clear warnings if they are absent. [1] [2] [3]
  • Updated documentation in .rhiza/tests/README.md to reflect the new location for benchmarks (tests/benchmarks/).

Test Utilities Refactor:

  • Removed the shared test_utils.py and instead defined helper functions (run_make, strip_ansi, setup_rhiza_git_repo) directly in each relevant conftest.py file for api and sync tests. This eliminates the need for relative imports and ensures all test modules are self-contained. [1] [2] [3] [4]
  • Updated test imports and usage to reference the new utility locations, and adjusted test assertions to expect uv run pytest instead of .venv/bin/python -m pytest. [1] [2] [3] [4]

These changes collectively improve the structure, maintainability, and reliability of the project's test and benchmarking infrastructure.

@HarryCampion HarryCampion requested a review from tschm February 7, 2026 20:46
@HarryCampion HarryCampion marked this pull request as ready for review February 7, 2026 20:53
@HarryCampion
Copy link
Collaborator Author

@tschm i will resolve conflicts but codefactor seems a bit buggy?

@tschm
Copy link
Member

tschm commented Feb 8, 2026

Yes, that's a weird CodeFactor problem. No issue and yet broken...

tschm and others added 2 commits February 8, 2026 16:09
Adds a new GitHub Actions workflow that verifies CodeFactor is functioning
and accessible. The workflow fetches the CodeFactor badge, extracts the
current grade, and fails if the service is unreachable or broken. This
ensures CodeFactor issues are surfaced in CI/CD rather than going unnoticed.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
CodeFactor is optional and not used in private repos, so removing the
CI check to avoid failures where CodeFactor isn't configured.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants