Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: fix/improve usage of "pending" #11184

Closed
wants to merge 13 commits into from
Closed

Conversation

blueyed
Copy link
Contributor

@blueyed blueyed commented Oct 8, 2019

No description provided.

@blueyed blueyed added the test label Oct 8, 2019
@@ -182,7 +182,7 @@ describe('server -> client', function()
describe('recursive (child) nvim client', function()
if helpers.isCI('travis') and helpers.is_os('mac') then
-- XXX: Hangs Travis macOS since e9061117a5b8f195c3f26a5cb94e18ddd7752d86.
pending("[Hangs on Travis macOS. #5002]", function() end)
pending("[Hangs on Travis macOS. #5002]")
Copy link
Member

@justinmk justinmk Oct 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless something changed in busted, this won't work. The function argument is needed.

Same for the other cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two cases for pending: 1) when using it instead of it, and 2) when using it inside of a test, where it will stop the test there then also.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh.. this is actually wrong, yes.
But "only" in that it would not display it as skipped then.

I think it is better to replace it with pending in those cases, so that the number of tests is the same, and there is no special no-op test that gets skipped, but is not available in general.
Also e.g. test filters etc would not find the existing (skipped) ones then.

@justinmk
Copy link
Member

justinmk commented Oct 17, 2019

unless something changed in busted, this won't work. The function argument is needed.

Doesn't seem like the problem is fixed. For example, the build log of the travis macOS build on master shows 17 skipped tests, including the [Hangs on Travis macOS. #5002] item. Whereas the build in this PR shows 15 skipped tests, and is missing these 2 skip messages:

�[1m�[33m[ SKIPPED  ]�[0m�[0m �[36mtest/functional/api/server_requests_spec.lua�[0m @ �[36m185�[0m: �[1mserver -> client recursive (child) nvim client [Hangs on Travis macOS. #5002]�[0m

�[1m�[33m[ SKIPPED  ]�[0m�[0m �[36mtest/functional/legacy/077_mf_hash_grow_spec.lua�[0m @ �[36m23�[0m: �[1mmf_hash_grow() was not tested because cksum was not found�[0m

local describe = describe
clear()
if missing_provider('node') then
describe = function(desc, ...) pending(desc.." (Missing nodejs host, or nodejs version is too old)", ...) end
Copy link
Member

@justinmk justinmk Oct 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this better than the old code? local describe = describe is awkward, and in general the new pattern is less obvious/clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better since it keeps the number of tests etc (see also previous comment)
(for reference lunarmodules/busted#610)

before_each(function()
clear()
command('python import vim')
end)
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this if-else better than the old return approach?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reason as before: to register the tests really.
The if/else is needed here to not use the before_each with the test that is only done when the provider is missing.

put ='case3: failed'
else
close
put ='case3: ok'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

? The whitespace is part of the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not fail though with it being removed. But have not really paid attention / looked at the test.

@justinmk
Copy link
Member

This doesn't seem worth the cost.

  • churn
  • more difficult patterns to remember and understand

blueyed added a commit to blueyed/neovim that referenced this pull request Oct 17, 2019
@blueyed
Copy link
Contributor Author

blueyed commented Oct 17, 2019

Yeah, it's quite messy indeed.

Created #11247 for clear misuse.
I think that helps using it better in the future also, given that patterns are often copied etc.

@blueyed blueyed closed this Oct 17, 2019
@blueyed blueyed deleted the fix-pending branch October 17, 2019 20:29
blueyed added a commit that referenced this pull request Oct 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants