Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to update snapshots with unittest? #51

Open
codeocelot opened this issue Jun 18, 2018 · 7 comments · May be fixed by #144
Open

How to update snapshots with unittest? #51

codeocelot opened this issue Jun 18, 2018 · 7 comments · May be fixed by #144
Labels

Comments

@codeocelot
Copy link

Appending --update-snapshot as is done with nosetest, etc throws ImportError: No module named --update-snapshot. The docs are unclear how exactly to pass this param when using unittest. Is this features supported when using unittest?

@paulmelnikow
Copy link
Collaborator

Not sure if there is a way to do this, though if there is I'd welcome a PR to add it to the documentation.

@medmunds
Copy link
Collaborator

I don't believe --update-snapshot is implemented when using raw unitttest. Unittest doesn't seem to have a way to add new command-line options for unittest.main().

You could implement it yourself by setting snapshot_should_update = True on the snapshottest.unittest.TestCase class when you want to update snapshots. Something like this (warning, untested code):

if __name__ == "__main__":
    import snapshottest.unittest
    import sys
    argv = sys.argv
    if "--update-snapshot" in argv:
        snapshottest.unittest.TestCase.snapshot_should_update = True
        argv = argv.copy()  # (don't modify sys.argv itself)
        argv.remove("--update-snapshot")
    unittest.main(argv=argv)

It might make sense to add a new snapshottest.unitttest.main() providing this functionality, which people could use in place of unittest.main().

@paulmelnikow
Copy link
Collaborator

paulmelnikow commented Sep 30, 2020

An alternative could be to set it in an environment variable. What do you think about that approach?

@medmunds
Copy link
Collaborator

I'm of mixed opinions on using an environment variable to request snapshot updates:

  • Pro: it would work with python -m unittest discover, which is another common way to invoke unittest.
  • Con: it's different from all of the other test packages, which support --update-snapshot on the command line.
  • Con (or Pro?): people would probably expect that environment variable to work with the other test packages then, too.

I think I'd lean toward providing the unittest.main() override, to parallel the other test packages. (Switching your unittest's __main__ is the equivalent of switching your Django TEST_RUNNER.) And then as a separate enhancement, also adding an environment variable supported for all the test packages. (And working out how that interacts with CI environments—I saw there was some activity around snapshot updates in CI, but haven't looked at the discussion yet.)

Of course, the other way to achieve this is deleting your existing snapshot files and re-running the tests, which will generate new snapshots. Maybe just document that for unittest?

@medmunds
Copy link
Collaborator

the other way to achieve this is deleting your existing snapshot files and re-running the tests, which will generate new snapshots

Ah, I see, that's going away in #73. So I guess this does need to get addressed some other way, or it'll be really hard for unittest users to get started with snapshots.

@davidshepherd7
Copy link
Collaborator

I added an environment variable option to #112 so that it doesn't make things worse for unittest users.

I think it would be better if someone who uses and understands unittest added a main override if we decide we do want one.

@medmunds
Copy link
Collaborator

I'm much more familiar with Django's unittest wrapper (which has its own test runner and command-line handling), but could take a shot at a main override.

Just realized that raw unittest is probably also missing the end-of-test-session handling (delete unvisited snapshots and report summary), which are coded separately for each of the other three test frameworks.

medmunds added a commit to medmunds/snapshottest that referenced this issue Oct 6, 2020
Add test-session-level features for raw unittest framework:
* Support --snapshot-update command line option
* Issue "SnapshotTest summary" at end of test run
* Remove unused snapshots when in update mode

This also adds snapshottest-augmented versions of the various
ways to run unittest:
* `python -m snapshottest ...` parallels `python -m unittest ...`
  (via module-level `__main__.py`).
* `snapshottest.main()` wraps `unittest.main()`.
* `SnapshotTestRunnerMixin` can be used with other unittest
  TestRunner classes.

Closes syrusakbary#51
medmunds added a commit to medmunds/snapshottest that referenced this issue Oct 6, 2020
Add test-session-level features for raw unittest framework:
* Support --snapshot-update command line option
* Issue "SnapshotTest summary" at end of test run
* Remove unused snapshots when in update mode

This also adds snapshottest-augmented versions of the various
ways to run unittest:
* `python -m snapshottest ...` parallels `python -m unittest ...`
  (via module-level `__main__.py`).
* `snapshottest.main()` wraps `unittest.main()`.
* `SnapshotTestRunnerMixin` can be used with other unittest
  TestRunner classes.

Closes syrusakbary#51
@medmunds medmunds linked a pull request Oct 6, 2020 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants