Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Unit and integration tests using GitHub actions #94

Open
szabo137 opened this issue Nov 7, 2024 · 0 comments
Open

[Proposal] Unit and integration tests using GitHub actions #94

szabo137 opened this issue Nov 7, 2024 · 0 comments
Assignees

Comments

@szabo137
Copy link
Member

szabo137 commented Nov 7, 2024

Problem statement

Our unit and integration tests run automatically using GitLab-CI on self-hosted runners on Gitlab.com via pull/push mirror or a dedicated bot. With this issue, we want to evaluate this setup and propose future updates.

Pros

  • gitlab-ci allows dynamic generation of job scrips, which enable convenient implementations of the integration test generation
  • easy inclusion of GPU tests (because we have runners with the needed hardware)
  • one test pipeline, which includes unit and integration tests, which gives a nice overview of the status of all tests in one place.

Cons

  • switching to another platform (here Gitlab.com) and limited hardware creates an overhead for all test triggers. This is especially critical for unit tests because they should provide the test information rapidly.
  • Convenient secondary services (e.g. Codecov, CoverageAll) are not possible or cumbersome to use. Especially the post of Codecov-reports directly into the pull-request increases developer ergonomics a lot.

Proposed solution

We run the unit and integration tests for every PR using GitHub actions and use GitLab-CI only for GPU tests (which by themselves have unit and integration tests). This would most probably decrease the time for the first test report and allow the native usage of Codecov and CoverageAll (both have native support for GitHub actions and GitHub itself).

Since the services in mind only use the unit tests, it is also conceivable to just move the unit tests to GitHub actions and leave the integration tests in the GitLab-CI pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants