Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for generated code behaviour #252

Open
n-osborne opened this issue Sep 10, 2024 · 0 comments
Open

Add tests for generated code behaviour #252

n-osborne opened this issue Sep 10, 2024 · 0 comments

Comments

@n-osborne
Copy link
Collaborator

Most of our tests are expect tests regarding the generated code.
The github workflow includes the dune build @launchtests call that actually run the generated code. This expects the test to be successful.

There is nothing checking the behaviour of the generated code that report an error in the tested module.

#248 was indeed spotted relatively late after the introduction of the concerned feature (and the bug) due to the lack of such tests.

We want to test:

  • the generated code spots the bugs (we are relatively confident about that one)
  • the display of the explanation of the bug: the Gospel term that is violated
  • the display of the runnable scenario (the commands, the returned and the expected values)

We should have some expect tests in which generated code for some buggy modules (examples of different failures) is run.

/!\ in order to have a reproducible output, we should run the generated code with a given seed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant