-
-
Notifications
You must be signed in to change notification settings - Fork 312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add codecov and run android tests via travis #526
Conversation
Codecov Report
@@ Coverage Diff @@
## master #526 +/- ##
=========================================
Coverage ? 31.86%
=========================================
Files ? 64
Lines ? 4852
Branches ? 655
=========================================
Hits ? 1546
Misses ? 3087
Partials ? 219 Continue to review full report at Codecov.
|
Travis build failed because of the failing espresso ui test. Fix is in #584 |
@mueller-ma Only the links to old Travis jobs are now broken - new ones are already correct as you can see on https://travis-ci.org/openhab/openhab-android |
So, I've run the same test suite locally and it succeeded. However, I used api level 27, travis uses 22 (I assume this is intentional?), however, this shouldn't make any difference. On the other hand, when running master, I also have only 4 tests: And org.openhab.habdroid.ui.BasicWidgetTest > openHABMainActivityTest[test(AVD) - 5.1.1] FAILED should've failed for 2 other tests, too, if there's really no demo sitemap. I currently rerun the tests with 5.1.1 and will checkout this pull request to maybe get it reproduced locally :) |
I'm planning to create a build matrix in a different PR after this one is merged. Than tests run on multiple APIs. |
Ok, so running the same test suite on Android 5.1.1, the view Hierarchy is slightly different. That's what you get on API level 27 currently:
And that's what you get on API level 22:
So, both AppCompatTextView with the So, the structure is slightly different, but the position of the demo is not 0 in api level 22 anymore, but 1. I think we can work around this by simply asserting, that demo is there and wrapped by a recycle list view? |
Yes, we can do that. All the atChild() are autogenerated by the espresso test recorder. |
See #592 |
@FlorianSW Tests are still failing |
They're not failing for me locally. Can you check that, too? |
Also works for me locally via Android Studio. |
Ran it again and now only one test failes with |
Hmm, possible. I'll submit a PR shortly, which will extract this test from the BasicWidgetTest (as it's not a widget test at all, and it makes it easier to skip it), which checks, if Voice recognition is supported/available on the device before running the test suite. With that we can make sure, that voice service is really not available on the AVD and probably change that in some way. |
android: | ||
components: | ||
- tools | ||
- build-tools-25.0.2 | ||
- platform-tools | ||
- tools | ||
- android-22 | ||
- sys-img-armeabi-v7a-android-22 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about adding some extras, which should probably enable voice recognition? However, this is not based on facts, just an idea :P I'm also not sure, which of these extras are already included :/ Also: We should first somehow make sure, that it's related to the play services :D
- extra-google-google_play_services
- extra-google-m2repository
- extra-android-m2repository
- extra-android-support
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we somehow check a build with these extras to see, if the voice recognition test is still skipped?
PR I mean: #609 |
Signed-off-by: mueller-ma <[email protected]>
Signed-off-by: mueller-ma <[email protected]>
|
Ok, could it be possible, that the colorpicker simply isn't in the view of the user? The colorpicker check uses a ViewInteraction to check, if it is display, instead of looking at the data and checking, that the picker is present. So, when a device, let's say WVGA with 480x800[1] pixel, the colorpicker is not in the view when navigating to the Widget Overview (selecting Widget Overview, btw., wouldn't pass the isDisplayedcheck, too, however, it already uses a data interaction to click on an element, which is not visible to the user already). This is the only thing I can think of at the moment. [1] Which is, what travis booted up:
|
@kaikreuzer Can you use @openhab-bot to add this repo on https://codecov.io/ ? |
I've logged in as openhab-bot and granted access to Github to codecov. When being logged in, I can see the page https://codecov.io/gh/openhab/openhab-android. |
Nothing, I'm going to merge this PR when we resolved the failing ci. |
# Emulator Management: Create, Start and Wait | ||
before_script: | ||
- android list device | ||
- echo no | android create avd --force -n test -t android-22 --abi armeabi-v7a -d "5.4in FWVGA" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, this doesn't solve the problem itself, it just works around it :/ What happens, if the demo sitemap changes slightly and the widgets we use in this test are again outside of the view?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a check for "exists, but not visible"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems there isn't :/ Then this should be fine for now :D
@kaikreuzer Can you check if codecov is setup correctly? On my fork, where I activeted it, codecov shows up a github check: https://github.com/mueller-ma-bot/openhab.android/pull/1 |
@mueller-ma I've just checked: The webhook wasn't yet created, I have done so now. |
Ok, we will see. |
#619 is a new PR, but it doesnt have the codecov check. Maybe you try to install codecov again via https://github.com/integration/codecov |
Installing it that way isn't possible: Note that openhab-bot does not own any repos. If I create a fork of openhab-android, I can enable it there, but that obviously won't help us. |
Can you set @openhab-bot as team bot? |
I've now installed the official Github integration as myself to the org account and activated it on this repo - I hope this now works! |
See this PR for an example: https://github.com/mueller-ma-bot/openhab.android/pull/1
I also tested Sputnik there, but this is not included in this PR.
@digitaldan This is just the config for codecov. To add it, go to their website and login via github.
Hopefully this will help to increase the code coverage. Current is ~6%