How to Run Tests

Testing in local environment

First, you must install pytest-astropy and pytest-qt in your virtual environment using pip install pytest-astropy pytest-qt.

Second, to run the tests:

• Run the command ./setup.py test (or python setup.py test) from the top-level directory

Note, when you run the test command you will get errors if the cubeviz instance that pops up is not in the forefront. (I tried running the tests and then went to another window and started typing and there were a bunch of errors in the tests. Re-ran with the cubeviz instance in the foreground and all was good.)

Reproducing CI tests using tox

tox is a test automation tool that makes it easy to define and create reproducible environments when running tests. PR #474 introduced the use of tox for testing on CI infrastructure. The primary purpose for this is to make it easier for developers to reproduce CI test failures locally.

In order to run tox locally you need a conda environment with the tox and tox-conda packages installed. Neither of these packages are distributed on conda channels, so you must use pip to install them.

Once tox and tox-conda are installed in your conda environment, you can use tox to run the test suite. The tox.ini file at the top level of the repository defines different test environments. When running tox, you should specify which environment to use. For example, the following command would run the test suite in an environment with Py36:

$ tox -e py36

Equivalently, you can use the TOXENV environment variable:

$ TOXENV=py36 tox

The tox.ini files defines dependencies that should be installed into the test environment (both from pip and conda). When the command is run, tox performs the following steps:

• Creates a new test environment (if this one doesn't exist already)
• Installs the dependencies specified in tox.ini
• Builds a cubeviz source distribution
• Installs cubeviz it in the test environment
• Runs the test command specified in tox.ini

Since tox.ini fully specifies environments to be used for testing, it is no longer necessary to do this in the CI configuration files. Instead, we simply need to make sure that tox and tox-conda are installed, and then each job in the test matrix simply calls tox with a different test environment.

This means that if we need to reproduce a CI job locally, we can simply run tox with the same arguments that were used in the test matrix.