Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Device Farm #2831

Merged
merged 23 commits into from
Mar 30, 2022
Merged

AWS Device Farm #2831

merged 23 commits into from
Mar 30, 2022

Conversation

fealebenpae
Copy link
Member

@fealebenpae fealebenpae commented Mar 3, 2022

Description

Run Android tests on AWS Device Farm physical devices.

@fealebenpae fealebenpae force-pushed the yg/android-devicefarm branch from 47a79f1 to b27362d Compare March 3, 2022 18:06
@cla-bot cla-bot bot added the cla: yes label Mar 3, 2022
@coveralls
Copy link

coveralls commented Mar 3, 2022

Pull Request Test Coverage Report for Build 2061381366

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 10 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.04%) to 82.938%

Files with Coverage Reduction New Missed Lines %
Realm/Realm/Handles/SessionHandle.cs 10 85.06%
Totals Coverage Status
Change from base Build 2029221850: -0.04%
Covered Lines: 5541
Relevant Lines: 6565

💛 - Coveralls

@fealebenpae fealebenpae force-pushed the yg/android-devicefarm branch from b27362d to ab05f32 Compare March 3, 2022 18:24
@fealebenpae fealebenpae force-pushed the yg/android-devicefarm branch 4 times, most recently from 0d138fc to dc14fd2 Compare March 4, 2022 00:34
@fealebenpae fealebenpae force-pushed the yg/android-devicefarm branch from dc14fd2 to c3f7dd0 Compare March 4, 2022 01:17
@fealebenpae fealebenpae force-pushed the yg/android-devicefarm branch from cabdbdc to 00bb2f8 Compare March 23, 2022 16:54
@fealebenpae fealebenpae reopened this Mar 23, 2022
@fealebenpae fealebenpae requested a review from nirinchev March 29, 2022 23:03
@fealebenpae fealebenpae self-assigned this Mar 29, 2022
Copy link
Member

@nirinchev nirinchev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, it seems fine to me, though I'd prefer to move a lot of the complexity to the device farm action as it seems counterproductive to maintain it across multiple repositories .

env:
REALM_DISABLE_ANALYTICS: true
DOTNET_NOLOGO: true
jobs:
run-tests:
runs-on: macos-latest
runs-on: windows-latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this run on Linux instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't build Xamarin.Android projects on Linux.

if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@v2
run: msbuild Tests/Tests.Android -t:SignAndroidPackage -p:Configuration=Release -restore -p:RestoreConfigFile=Tests/Test.NuGet.Config -p:UseRealmNupkgsWithVersion=${{ inputs.version }} -p:AndroidSupportedAbis=armeabi-v7a -p:AndroidUseSharedRuntime=False -p:EmbedAssembliesIntoApk=True
- name: Configure AWS Credentials
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to configure AWS credentials here? Shouldn't this be done by the device farm action?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a convention with actions using the AWS SDK. Besides, in this specific case I need to use some AWS PowerShell cmdlets and they also need the same credentials.

Comment on lines +78 to +84
app_type: ANDROID_APP
test_type: APPIUM_PYTHON
test_package_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/sample_env_python3.zip
test_package_type: APPIUM_PYTHON_TEST_PACKAGE
test_spec_file: test_spec.yaml
test_spec_type: APPIUM_PYTHON_TEST_SPEC
remote_src: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't seem like something that should be configurable. It's fine to leave them like this for now, but I'd rather set them as defaults in the action and not have us specify them every time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll wait until I have a working version for iOS as well, or until we need to integrate this into other SDKs. It's not unlikely that other SDKs will use a different test package or spec. But I agree, this can use a refactoring on a second pass.

Comment on lines +107 to +120
- run: |
Expand-Archive 'Customer Artifacts.zip' -DestinationPath artifacts
Import-Module AWSPowerShell
$jobs = Get-DFJobList -Arn ${{ steps.run_tests.outputs.arn }}
$suites = Get-DFSuiteList -Arn $jobs[0].Arn
$artifacts = Get-DFArtifactList -Arn $suites[1].Arn -Type File | Where-Object { $_.Name -EQ "Logcat" }
echo "::group::Logcat"
Invoke-WebRequest -Uri $artifacts[0].Url | Select-Object -Expand RawContent
echo "::endgroup::"
name: Fetch test artifacts
- run: |
echo "::group::Data"
echo (ConvertFrom-Json '${{ steps.run_tests.outputs.data }}' | ConvertTo-Json)
echo "::endgroup::"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This too seems like something that should be done internally in the device farm action rather than here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I filed realm/aws-devicefarm#6 to track this. It's something @bwachter and I already discussed.

@fealebenpae fealebenpae merged commit 7867459 into main Mar 30, 2022
@fealebenpae fealebenpae deleted the yg/android-devicefarm branch March 30, 2022 12:53
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants