Style & Formatting

This section outlines OtterTune's style/formatting guidelines and helper scripts.

Style Guides

We use Google's style/format conventions:

• Google Python Style Guide
• Google Java Style Guide

Source Validator Script

We provide a source code validation script that performs the following checks:

• check_pylint: Runs pylint on the Python source code

• check_pycodestyle: Runs pycodestyle on the Python source code

• check_java_checkstyle: Runs checkstyle on the Java source code

• check_illegal_patterns: Checks that the source code does not use any illegal statements (e.g., print)

• check_header: Checks that all source files have the correct header

Our source validation script returns 0 on success and -1 on failure. We execute this script on travis-ci so failing any of its checks will also cause the travis-ci build to fail.

Usage (python script/validators/source_validator.py -h):

usage: source_validator.py [-h] [--staged-files] [PATH [PATH ...]]

Validate OtterTune's source code

positional arguments:
  PATH            Files or directories to (recursively) validate

optional arguments:
  -h, --help      show this help message and exit
  --staged-files  Apply the selected action(s) to all staged files (git)

Usage examples:

# Validate all source files in the ottertune repo:
python script/validators/source_validator.py

# Validate all files in the analysis directory and the file fabfile.py
python script/validators/source_validator.py server/analysis server/website/fabfile.py

# Validate all files staged for commit (git)
python script/validators/source_validator.py --staged-files

Formatter Script

We provide a formatting script that automatically formats source code files and adds headers. For python, it uses autopep8 to fix the formatting. Note that since autopep8 uses pycodestyle to detect the style errors it should fix all (or most) of them; however, it does not use pylint so it may not fix all of its reported errors. For Java, it uses google-java-format to automatically format the source files. Similar to Python, google-java-format will fix most but possibly not all of the errors reported by checkstyle.

Usage (python script/formatting/formatter.py -h):

usage: formatter.py [-h] [--no-update-header] [--no-format-code]
                    [--staged-files]
                    [PATH [PATH ...]]

Formats Python/Java source files in place

positional arguments:
  PATH                Files or directories to (recursively) apply the actions
                      to

optional arguments:
  -h, --help          show this help message and exit
  --no-update-header  Do not update the Python/Java source file headers
  --no-format-code    Do not format the Python/Java source files
  --staged-files      Apply the selected action(s) to all staged files (git)

Pre-commit Git Hook

We provide a pre-commit git hook script that, when enabled, will run the source validation script on any modified files staged for commit. The commit will succeed only if the source validation script passes.

To enable, do the following:

cd .git/hooks
ln -s ../../script/git-hooks/pre-commit ./pre-commit

Pylint

We use pylint to help analyze our Python source code using a custom configuration file.

To run pylint on its own, use:

pylint --rcfile=script/formatting/config/pylintrc [path(s)]

Pylint is thorough, but sometimes it reports false errors that you will need to disable. You might also want pylint to ignore a specific error on a line of code for other reasons.

To disable an error message on a particular line of code, you can add an inline comment of the form: # pylint: disable=[message-code].

In the example below, pylint is outputing a (false) message that os.walk(dir_path) is not iterable. We disable it using the not-an-iterable message code:

def format_dir(dir_path, update_header, format_code):
    for subdir, _, files in os.walk(dir_path):  # pylint: disable=not-an-iterable
        for file_path in files:
            file_path = subdir + os.path.sep + file_path
            format_file(file_path, update_header, format_code)

Pycodestyle

We also use pycodestyle to help analyze our Python source code using a custom configuration file.

To run pycodestyle on its own, use:

pycodestyle --config=script/formatting/config/pycodestyle [path(s)]

Like pylint, you can also ignore an error message on a line by using an inline comment of the form: # pycodestyle: disable=[error_code].

Checkstyle

We use checkstyle to help analyze our Java source code using a custom configuration file.

To run checkstyle on its own, you must first make sure the jar file is downloaded. Starting from the top ottertune directory, run:

cd controller
gradle downloadJars
cd ..

Then to run checkstyle, use:

java -jar controller/build/libs/checkstyle-8.8-all.jar -c script/formatting/config/google_checks.xml script/formatting/config/google_checks.xml [path(s)]