Skip to content

Commit

Permalink
ci: split UI tests (#1460)
Browse files Browse the repository at this point in the history
**Issue number:** N/A

### PR Type

**What kind of change does this PR introduce?**
* [ ] Feature
* [ ] Bug Fix
* [ ] Refactoring (no functional or API changes)
* [ ] Documentation Update
* [x] Maintenance (dependency updates, CI, etc.)

## Summary

### Changes

Splitting UI tests in GitHub Actions pipeline.

According to the run from this PR - the whole pipeline time dropped 50%.

I checked the total amount of test cases run and it is the same as in
the previous approach.

### User experience

N/A

## Checklist

If an item doesn't apply to your changes, leave it unchecked.

* [x] I have performed a self-review of this change according to the
[development
guidelines](https://splunk.github.io/addonfactory-ucc-generator/contributing/#development-guidelines)
* [ ] Tests have been added/modified to cover the changes [(testing
doc)](https://splunk.github.io/addonfactory-ucc-generator/contributing/#build-and-test)
* [ ] Changes are documented
* [x] PR title and description follows the [contributing
principles](https://splunk.github.io/addonfactory-ucc-generator/contributing/#pull-requests)
  • Loading branch information
artemrys authored Nov 14, 2024
1 parent a23e345 commit d4176e2
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 13 deletions.
27 changes: 15 additions & 12 deletions .github/workflows/build-test-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ jobs:
path: Splunk_TA_UCCExample*.tar.gz

test-ui:
name: test-ui Splunk ${{ matrix.splunk.version }} -m ${{ matrix.test-mark }}
name: test-ui Splunk ${{ matrix.splunk.version }} -group ${{ matrix.test-group }}
runs-on: ubuntu-22.04
permissions:
id-token: write
Expand All @@ -243,14 +243,17 @@ jobs:
fail-fast: false
matrix:
splunk: ${{ fromJson(needs.meta.outputs.matrix_supportedSplunk) }}
test-mark:
- "logging"
- "proxy"
- "account"
- "custom"
- "alert"
- "input"
- "configuration"
test-group:
- "1"
- "2"
- "3"
- "4"
- "5"
- "6"
- "7"
- "8"
- "9"
- "10"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
Expand All @@ -272,16 +275,16 @@ jobs:
./scripts/run_splunk.sh ${{ matrix.splunk.version }}
until curl -Lsk "https://localhost:8088/services/collector/health" &>/dev/null ; do echo -n "Waiting for HEC-" && sleep 5 ; done
timeout-minutes: 5
- run: poetry run pytest tests/ui -m "${{ matrix.test-mark }}" --headless --junitxml=test-results/junit.xml
- run: poetry run pytest tests/ui --test-group-count 10 --test-group ${{ matrix.test-group }} --test-group-random-seed 123456 --headless --junitxml=test-results/junit.xml
- uses: actions/upload-artifact@v4
if: success() || failure()
with:
name: test-results-ui-${{ matrix.splunk.version }}-${{ matrix.test-mark }}
name: test-results-ui-${{ matrix.splunk.version }}-${{ matrix.test-group }}
path: test-results/*
- uses: dorny/test-reporter@v1
if: success() || failure()
with:
name: test-report-ui-${{ matrix.splunk.version }}-${{ matrix.test-mark }}
name: test-report-ui-${{ matrix.splunk.version }}-${{ matrix.test-group }}
path: "test-results/*.xml"
reporter: java-junit

Expand Down
16 changes: 15 additions & 1 deletion poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ mkdocs-print-site-plugin = "^2.3.6"
pytest-cov = "^4.0.0"
covdefaults = "^2.3.0"
xmldiff = "^2.6.3"
pytest-split-tests = "^1.1.0"

[tool.poetry.scripts]
ucc-gen="splunk_add_on_ucc_framework.main:main"
Expand Down

0 comments on commit d4176e2

Please sign in to comment.