-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When a test framework setup fails, no test are run but the run succeed #2098
Comments
This is by design (as in we're using ginkgo incorrectly) - we run framework initialisation in a container node, so any failing assertions there will fail at the spec tree construction phase, not when running specs, see https://onsi.github.io/ginkgo/#no-assertions-in-container-nodes. Iirc putting GinkgoRecover there only causes it to (unintentionally from our perspective) pass silently on a failed assertion - if you remove it you'll see that the spec tree construction fails. GinkgoRecover shouldn't be called in the container nodes in the first place, see e.g. onsi/ginkgo#931 (comment). Imo to properly fix this we'd have to move all initialisation in the framework from the container nodes to setup nodes. I think I actually tried to do this at some point but it required quite a few changes given how the framework and tests are set up now. |
yeah, I guess we should put stuff that can fail (init) into a beforeEach the weird thing is that is works if not all nodes fail... |
Can you give an example? I'm not sure what that means |
The Scylla Operator project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
/lifecycle stale |
The Scylla Operator project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
/lifecycle rotten |
/remove-lifecycle rotten |
When any assertion in the
Framework
fails, no test are executed but the run succeeds. It can be simulated by minting the framework likethis go through ginkgo.Recover and calls the fail handler correctly but my suspicion is that when no test gets registered is forgets to check the fail handler. I've tried failing the individual tests before the framework creation directly and that has correctly failed the run.
The text was updated successfully, but these errors were encountered: