Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: New Python-based terraform_fmt hook to better support Windows #652

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
tests/results/*
__pycache__/
*.py[cod]
node_modules/
68 changes: 66 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ repos:

# Dockerfile linter
- repo: https://github.com/hadolint/hadolint
rev: v2.12.1-beta
rev: v2.13.0-beta
hooks:
- id: hadolint
args: [
Expand All @@ -54,8 +54,72 @@ repos:

# JSON5 Linter
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
rev: v3.1.0
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved
hooks:
- id: prettier
# https://prettier.io/docs/en/options.html#parser
files: '.json5$'


##########
# PYTHON #
##########

- repo: https://github.com/asottile/reorder_python_imports
rev: v3.12.0
hooks:
- id: reorder-python-imports

- repo: https://github.com/asottile/add-trailing-comma
rev: v3.1.0
hooks:
- id: add-trailing-comma

- repo: https://github.com/pre-commit/mirrors-autopep8
rev: v2.0.4
hooks:
- id: autopep8
args:
- -i
- --max-line-length=100

# Usage: http://pylint.pycqa.org/en/latest/user_guide/message-control.html
- repo: https://github.com/PyCQA/pylint
rev: v3.1.0
hooks:
- id: pylint
args:
- --disable=import-error # E0401. Locally you could not have all imports.
- --disable=fixme # W0511. 'TODO' notations.
- --disable=logging-fstring-interpolation # Conflict with "use a single formatting" WPS323
- --disable=ungrouped-imports # ignore `if TYPE_CHECKING` case. Other do reorder-python-imports

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.10.0
hooks:
- id: mypy
args: [
--ignore-missing-imports,
--disallow-untyped-calls,
--warn-redundant-casts,
]

- repo: https://github.com/pycqa/flake8.git
rev: 7.0.0
hooks:
- id: flake8
additional_dependencies:
- flake8-2020
- flake8-docstrings
- flake8-pytest-style
- wemake-python-styleguide
args:
- --max-returns=2 # Default settings
- --max-arguments=4 # Default settings
# https://www.flake8rules.com/
# https://wemake-python-stylegui.de/en/latest/pages/usage/violations/index.html
- --extend-ignore=
WPS305, <!-- Found `f` string -->
E501, <!-- line too long (> 79 characters). Use 100 -->
I, <!-- opt out of using isort in favor of reorder-python-imports -->
# RST, <!-- Conflict with DAR -->
8 changes: 8 additions & 0 deletions .pre-commit-hooks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@
files: (\.tf|\.tfvars)$
exclude: \.terraform/.*$

- id: terraform_fmt_py
name: Terraform fmt
description: Rewrites all Terraform configuration files to a canonical format.
entry: terraform_fmt
language: python
files: \.tf(vars)?$
exclude: \.terraform/.*$

- id: terraform_docs
name: Terraform docs
description: Inserts input and output documentation into README.md (using terraform-docs).
Expand Down
4 changes: 0 additions & 4 deletions hooks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +0,0 @@
print(
'`terraform_docs_replace` hook is DEPRECATED.'
'For migration instructions see https://github.com/antonbabenko/pre-commit-terraform/issues/248#issuecomment-1290829226'
)
108 changes: 108 additions & 0 deletions hooks/common.py
Copy link
Collaborator

@MaxymVlasov MaxymVlasov Apr 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's start a PR scope thread here:

How much do we want to be implemented in this PR?

1.1. Current common functional "as is"
1.2. Full common functional parity

2.1. Just terraform_fmt
2.2. All hooks fully which fully based on common functional

Choose one from 1. and 2.
The current state is 1.1. + 2.1.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.1 + 2.1 (+ each hook to be re-implemented with Python in scope of a separate PR)

ps: I don't get the difference between 1.1 and 1.2 to be honest... 🤷🏻

Copy link
Collaborator

@MaxymVlasov MaxymVlasov Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
"""
Here located common functions for hooks.

It not executed directly, but imported by other hooks.
"""
from __future__ import annotations

import argparse
import logging
import os
from collections.abc import Sequence

logger = logging.getLogger(__name__)


def setup_logging():
"""
Set up the logging configuration based on the value of the 'PCT_LOG' environment variable.

The 'PCT_LOG' environment variable determines the logging level to be used.
The available levels are:
- 'error': Only log error messages.
- 'warn' or 'warning': Log warning messages and above.
- 'info': Log informational messages and above.
- 'debug': Log debug messages and above.

If the 'PCT_LOG' environment variable is not set or has an invalid value,
the default logging level is 'warning'.

Returns:
None
"""
log_level = {
'error': logging.ERROR,
'warn': logging.WARNING,
'warning': logging.WARNING,
'info': logging.INFO,
'debug': logging.DEBUG,
}[os.environ.get('PCT_LOG', 'warning').lower()]

logging.basicConfig(level=log_level)


def parse_env_vars(env_var_strs: list[str]) -> dict[str, str]:
"""
Expand environment variables definition into their values in '--args'.

Args:
env_var_strs (list[str]): A list of environment variable strings in the format "name=value".

Returns:
dict[str, str]: A dictionary mapping variable names to their corresponding values.
"""
ret = {}
for env_var_str in env_var_strs:
name, env_var_value = env_var_str.split('=', 1)
if env_var_value.startswith('"') and env_var_value.endswith('"'):
env_var_value = env_var_value[1:-1]
ret[name] = env_var_value
return ret


def parse_cmdline(
argv: Sequence[str] | None = None,
) -> tuple[list[str], list[str], list[str], list[str], dict[str, str]]:
"""
Parse the command line arguments and return a tuple containing the parsed values.

Args:
argv (Sequence[str] | None): The command line arguments to parse.
If None, the arguments from sys.argv will be used.

Returns:
tuple[list[str], list[str], list[str], list[str], dict[str, str]]:
A tuple containing the parsed values:
- args (list[str]): The parsed arguments.
- hook_config (list[str]): The parsed hook configurations.
- files (list[str]): The parsed files.
- tf_init_args (list[str]): The parsed Terraform initialization arguments.
- env_var_dict (dict[str, str]): The parsed environment variables as a dictionary.
"""

parser = argparse.ArgumentParser(
add_help=False, # Allow the use of `-h` for compatibility with the Bash version of the hook
)
parser.add_argument('-a', '--args', action='append', help='Arguments')
parser.add_argument('-h', '--hook-config', action='append', help='Hook Config')
parser.add_argument('-i', '--init-args', '--tf-init-args', action='append', help='Init Args')
parser.add_argument('-e', '--envs', '--env-vars', action='append', help='Environment Variables')
parser.add_argument('FILES', nargs='*', help='Files')

parsed_args = parser.parse_args(argv)

args = parsed_args.args or []
hook_config = parsed_args.hook_config or []
files = parsed_args.FILES or []
tf_init_args = parsed_args.init_args or []
env_vars = parsed_args.envs or []

env_var_dict = parse_env_vars(env_vars)

if hook_config:
raise NotImplementedError('TODO: implement: hook_config')

if tf_init_args:
raise NotImplementedError('TODO: implement: tf_init_args')

return args, hook_config, files, tf_init_args, env_var_dict
46 changes: 32 additions & 14 deletions hooks/terraform_docs_replace.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
"""Deprecated hook to replace README.md with the output of terraform-docs."""
import argparse
import os
import subprocess
import sys

print(
'`terraform_docs_replace` hook is DEPRECATED.'
'For migration instructions see ' +
'https://github.com/antonbabenko/pre-commit-terraform/issues/248#issuecomment-1290829226',
)


def main(argv=None):
"""
TODO: Add docstring.

Args:
argv (list): List of command-line arguments (default: None)

Returns:
int: The return value indicating the success or failure of the function
"""
parser = argparse.ArgumentParser(
description="""Run terraform-docs on a set of files. Follows the standard convention of
pulling the documentation from main.tf in order to replace the entire
README.md file each time."""
README.md file each time.""",
)
parser.add_argument(
'--dest', dest='dest', default='README.md',
Expand All @@ -29,25 +45,27 @@ def main(argv=None):

dirs = []
for filename in args.filenames:
if (os.path.realpath(filename) not in dirs and
(filename.endswith(".tf") or filename.endswith(".tfvars"))):
if (
os.path.realpath(filename) not in dirs and
(filename.endswith('.tf') or filename.endswith('.tfvars'))
):
dirs.append(os.path.dirname(filename))

retval = 0

for dir in dirs:
for directory in dirs:
try:
procArgs = []
procArgs.append('terraform-docs')
proc_args = []
proc_args.append('terraform-docs')
if args.sort:
procArgs.append('--sort-by-required')
procArgs.append('md')
procArgs.append("./{dir}".format(dir=dir))
procArgs.append('>')
procArgs.append("./{dir}/{dest}".format(dir=dir, dest=args.dest))
subprocess.check_call(" ".join(procArgs), shell=True)
except subprocess.CalledProcessError as e:
print(e)
proc_args.append('--sort-by-required')
proc_args.append('md')
proc_args.append(f'./{directory}')
proc_args.append('>')
proc_args.append(f'./{directory}/{args.dest}')
subprocess.check_call(' '.join(proc_args), shell=True)
except subprocess.CalledProcessError as exeption:
print(exeption)
retval = 1
return retval

Expand Down
46 changes: 46 additions & 0 deletions hooks/terraform_fmt.py
ericfrederich marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
"""
Pre-commit hook for terraform fmt.
"""
from __future__ import annotations

import logging
import os
import shlex
import sys
from subprocess import PIPE
from subprocess import run
from typing import Sequence

from .common import parse_cmdline
from .common import setup_logging

logger = logging.getLogger(__name__)


def main(argv: Sequence[str] | None = None) -> int:

setup_logging()

logger.debug(sys.version_info)

args, hook_config, files, tf_init_args, env_vars = parse_cmdline(argv)

if os.environ.get('PRE_COMMIT_COLOR') == 'never':
args.append('-no-color')

cmd = ['terraform', 'fmt', *args, *files]

logger.info('calling %s', shlex.join(cmd))
logger.debug('env_vars: %r', env_vars)
logger.debug('args: %r', args)

completed_process = run(cmd, env={**os.environ, **env_vars}, text=True, stdout=PIPE)

if completed_process.stdout:
print(completed_process.stdout)

return completed_process.returncode


if __name__ == '__main__':
raise SystemExit(main())
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
entry_points={
'console_scripts': [
'terraform_docs_replace = hooks.terraform_docs_replace:main',
'terraform_fmt = hooks.terraform_fmt:main',
],
},
)
Loading