Error when running tests in Firebase Test Lab using Flank on API 30 and above

[ramzyabushaaban]

I'm submitting a ...

• [-] bug report

What is the current behavior?

When running the tests on my local emulator (API 30) I can see the Allure report files generated correctly with no problem, however, I get the below exception when running the same tests with the same configs in Firebase Test Lab using Flank (Pixel2 API 30)

androidx.test.services.storage.TestStorageException: No content provider registered for: content://androidx.test.services.storage.outputfiles/allure-results/72ee8821-c17d-4341-abb4-36ef6f1513cd-result.json. Are all test services apks installed? 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.services.storage.internal.TestStorageUtil.makeContentProviderClient(TestStorageUtil.java:108) 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.services.storage.internal.TestStorageUtil.getOutputStream(TestStorageUtil.java:88) 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.services.storage.TestStorage.openOutputFile(TestStorage.java:223) 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.services.storage.TestStorage.openOutputFile(TestStorage.java:203) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.android.writer.TestStorageResultsWriter$outputStreamResultsWriter$1.invoke(TestStorageResultsWriter.kt:16) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.android.writer.TestStorageResultsWriter$outputStreamResultsWriter$1.invoke(TestStorageResultsWriter.kt:15) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.kotlin.OutputStreamResultsWriter.write(OutputStreamResultsWriter.kt:23) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.android.writer.TestStorageResultsWriter.write(TestStorageResultsWriter.kt:20) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.kotlin.AllureLifecycle.writeTestCase(AllureLifecycle.kt:330) 07-04 17:57:18.561: E/TestRunner(12260): at io.qameta.allure.kotlin.junit4.AllureJunit4.testFinished(AllureJunit4.kt:60) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 07-04 17:57:18.561: E/TestRunner(12260): at io.victoralbertos.device_animation_test_rule.DeviceAnimationTestRule$1.evaluate(DeviceAnimationTestRule.java:49) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.rules.RunRules.evaluate(RunRules.java:20) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.Suite.runChild(Suite.java:128) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.Suite.runChild(Suite.java:27) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runner.JUnitCore.run(JUnitCore.java:137) 07-04 17:57:18.561: E/TestRunner(12260): at org.junit.runner.JUnitCore.run(JUnitCore.java:115) 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56) 07-04 17:57:18.561: E/TestRunner(12260): at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:395) 07-04 17:57:18.561: E/TestRunner(12260): at com.deliveryhero.pandora.test.PandoraTestRunner.onStart(PandoraTestRunner.kt:76) 07-04 17:57:18.561: E/TestRunner(12260): at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2205)

Please tell us about your environment:

My grade

testInstrumentationRunner = "io.qameta.allure.android.runners.AllureAndroidJUnitRunner" testInstrumentationRunnerArguments += mapOf("clearPackageData" to "true", "useTestStorageService" to "true")

I also use this property
allure.results.useTestStorage=true

Flank Configs
gcloud: results-bucket: android-ta results-dir: 139727 record-video: true timeout: 30m async: false client-details: network-profile: null results-history-name: null # Android gcloud app: /bitrise/deploy/app.apk test: /bitrise/deploy/apptest.apk additional-apks: auto-google-login: false use-orchestrator: true directories-to-pull: - /sdcard/ grant-permissions: all type: null other-files: scenario-numbers: scenario-labels: obb-files: obb-names: performance-metrics: false num-uniform-shards: null test-runner-class: null test-targets: - annotation de.foodora.android.util.CriticalTest robo-directives: robo-script: null device: - model: Pixel2 version: 30 locale: en_US orientation: portrait num-flaky-test-attempts: 1 test-targets-for-shard: fail-fast: false flank: max-test-shards: 8 shard-time: 30 num-test-runs: 1 smart-flank-gcs-path: gs://android-ta/android_app_ta.xml smart-flank-disable-upload: false default-test-time: 120.0 use-average-test-time-for-new-tests: false files-to-download: test-targets-always-run: disable-sharding: false project: [REDACTED] local-result-dir: results full-junit-result: false # Android Flank Yml keep-file-path: false additional-app-test-apks: run-timeout: -1 legacy-junit-result: false ignore-failed-tests: false output-style: multi disable-results-upload: false default-class-test-time: 240.0 disable-usage-statistics: false output-report: none skip-config-validation: false custom-sharding-json:

| Allure version | 2.4.0 |

[PrimankaDEN]

I hope it would be helpful for someone.
I think it is firebase problem. I faced with similar problem on microsoft appcenter. The reason is appcenter does not support orchestrator.
This content provider is part of https://mvnrepository.com/artifact/androidx.test.services/test-services You should install orchestrator.apk and test-services.apk to run tests. When you run tests on your local emulator, gradle script does it for you, but appcenter does not