Enable Playground tests on GitHub Actions CI workflow#14790
Draft
Enable Playground tests on GitHub Actions CI workflow#14790
Conversation
Add comprehensive documentation for the Azure DevOps public pipeline (azure-pipelines-public) covering pipeline architecture, Helix test categories, test archiving, test routing, and 9 known breakage patterns identified from real incidents in git history. The doc helps developers and agents understand what can break on AzDO since tests don't run on PRs and only run weekly. Includes a PR review checklist for AzDO safety. Also adds a brief reference in AGENTS.md under Continuous Integration pointing to the new doc. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dance - Fix /azp run command in azdo-tests.yml comment to match actual pipeline name - Remove shell command escaping section that contradicted working implementation - Remove DCP wait from post-commands (Linux-only, not cross-platform) - Renumber remaining Known Breakage Patterns sections Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add language identifiers to 4 fenced code blocks (MD040) - Fix manual trigger label from 'azdo-tests' to 'aspire-tests' in table - Update breakage patterns count from implicit 9 to explicit 8 - Use full path for azure-pipelines-public.yml in AGENTS.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a dedicated 'Playground Linux' job to the tests.yml workflow that: - Builds Aspire.Playground.Tests from source outside the repo - Uses built NuGet packages via the archive mechanism - Runs with Docker support on ubuntu-latest Changes: - Add buildOnTarget parameter to run-tests.yml for tests that need to build from source in the extracted archive - Add playground_tests job to tests.yml - Exclude Playground from integration tests matrix (dedicated job) - Fix missing TestFeature.cs and PathLookupHelper.cs in helix archive - Remove resolved FIXME comment in TestRunsheetBuilder.targets Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 14790Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 14790" |
The tests/.editorconfig suppresses CA2007 and CA1305 diagnostics for test projects. Without it in the archive, these warnings become errors when building with TreatWarningsAsErrors=true. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
🎬 CLI E2E Test RecordingsThe following terminal recordings are available for commit
📹 Recordings uploaded automatically from CI run #22520370561 |
Member
Author
|
/azp run aspire-tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
…ead of concrete type ContainerRegistryHook was injecting the concrete DistributedApplicationEventing type, but only the IDistributedApplicationEventing interface is registered in the DI container. This caused all tests to fail with a service resolution error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/azp run aspire-tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The ~15 container-based integration tests each take 2-10+ minutes. The previous 25m session timeout caused hang dumps mid-test, and with the 60m job timeout the full suite could not complete. Increased both GH Actions and Helix timeouts to 50m to accommodate the full test suite. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/azp run aspire-tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The 60m hardcoded job timeout is too short for Playground tests which need ~10m for setup/build plus 50m for running ~15 container-based integration tests. Added jobTimeout input (default 60m) and set Playground to 90m. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/azp run aspire-tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
3 tasks
The SqlServerEndToEnd test hangs for 55+ minutes on GH Actions because app.StartAsync() blocks on a container that never becomes ready. Reduced both --timeout and --hangdump-timeout to 10m (matching the default BlameHangTimeout used for in-repo tests). This ensures hanging tests are killed promptly instead of consuming the entire job timeout. Reverted the Helix timeout back to 25m and removed the 90m jobTimeout since tests should complete much faster with per-test hang detection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ation MTP --timeout and --hangdump-timeout don't reliably terminate hanging tests that produce continuous output (Kafka/Redis connection retries). Wrapped dotnet test with timeout --signal=TERM using testSessionTimeout as the hard deadline. Set Playground to 40m hard timeout with 10m MTP hang detection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR enables the Aspire.Playground.Tests to run outside the repo on the GitHub Actions CI workflow, using built NuGet packages instead of project references.
Changes
Archive fixes (
Aspire.Playground.Tests.csproj):TestFeature.csandPathLookupHelper.csto the PrepareForHelix archive items (they were referenced as Compile items in-repo but missing from the archive)RunOnGithubActionsLinux=falseto exclude Playground from the integration test matrix (it now has its own dedicated job)Workflow changes (
run-tests.yml):buildOnTargetinput parameter for tests that need to build from source in the extracted archive (not just run pre-built DLLs)buildOnTargetis trueTest orchestration (
tests.yml):playground_testsjob: Playground Linux (ubuntu-latest, 25m timeout)playground_teststo results job dependency listCleanup:
TestRunsheetBuilder.targetsabout Playground not running in out-of-repo modeValidation
Locally validated by:
PrepareForHelix=trueTestsRunningOutsideOfRepo=trueandBUILT_NUGETS_PATHset