Skip to content

Commit 81db08e

Browse files
dmlb2000ssbarnea
authored andcommitted
Fix #1788 add pre commit linter (#2161)
* Fix #1788 Add pre-commit to lint unit tests This adds a pre-commit verifier linter to run pre-commit on the molecule unit tests. Signed-off-by: David Brown <[email protected]> Update molecule/model/schema_v2.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/base.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update test/unit/verifier/lint/test_precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Missed one reference Signed-off-by: David Brown <[email protected]> Missed this rename as well Signed-off-by: David Brown <[email protected]> more unit test fixes Signed-off-by: David Brown <[email protected]> Add testinfra senerio with pre-commit This adds a senerio that can be run to show how pre-commit works with testinfra and an example pre-commit configuration file. Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update test/unit/verifier/lint/test_precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update test/unit/verifier/lint/test_precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> fix some of the suggestions to make things work Signed-off-by: David Brown <[email protected]> Should fix decentral1se issues. Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Update molecule/verifier/lint/precommit.py Co-Authored-By: dmlb2000 <[email protected]> Signed-off-by: David Brown <[email protected]> Hopefully to address the last thing? Signed-off-by: David Brown <[email protected]> Add pre-commit to tox deps Signed-off-by: David Brown <[email protected]> The rebase didn't merge quite right. Signed-off-by: David Brown <[email protected]> The rebase didn't merge quite right. Signed-off-by: David Brown <[email protected]> Fix some pre-commit linting issues Signed-off-by: David Brown <[email protected]> More pre-commit fixes Signed-off-by: David Brown <[email protected]> Try to get the schema right Signed-off-by: David Brown <[email protected]> Fix bug when molecule crash if env value is not a string (#2160) Fixes #2131 When configuring values in environment that are not strings, ensure to convert them to string. Signed-off-by: Sagi Shnaidman <[email protected]> Signed-off-by: Sorin Sbarnea <[email protected]> Add `--parallel` to functional test suite (#2155) * Mark tests as parallelizable * Add new parallelizable shard * Move test cleanup into Tox configuration * Adapt multi-node tests to parallelizable configuration * Ensure that strings are passed (py2 compat) * Adapt to new Black style formatting Add retry and back-off for dependency download (#2157) Closes #2114. Improve pytest support (#2144) * Fixes issue where `pytest --collect-only` went broken because on CI we always call it with specific folders. * Check reuses the virtualenv in order to avoid overloading disk with extra environments. * Repurposes metadata-check environment as a more generic check tox environment, now including regression check for pytest collection. Signed-off-by: Sorin Sbarnea <[email protected]> Signed-off-by: David Brown <[email protected]> * Fix recommendations on documentation. Signed-off-by: David Brown <[email protected]> * Add back pre-commit requires Signed-off-by: David Brown <[email protected]>
1 parent 6aecfc4 commit 81db08e

File tree

11 files changed

+443
-1
lines changed

11 files changed

+443
-1
lines changed

docs/configuration.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -412,3 +412,6 @@ Lint
412412

413413
.. autoclass:: molecule.verifier.lint.flake8.Flake8()
414414
:undoc-members:
415+
416+
.. autoclass:: molecule.verifier.lint.precommit.PreCommit()
417+
:undoc-members:

molecule/model/schema_v2.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,7 @@ def pre_validate_base_schema(env, keep_string):
141141
'molecule_env_var': True,
142142
'allowed': [
143143
'flake8',
144+
'pre-commit',
144145
'rubocop',
145146
'yamllint',
146147
'ansible-lint',
@@ -591,7 +592,9 @@ def pre_validate_base_schema(env, keep_string):
591592
'name': {'type': 'string', 'allowed': ['testinfra']},
592593
'lint': {
593594
'type': 'dict',
594-
'schema': {'name': {'type': 'string', 'allowed': ['flake8']}},
595+
'schema': {
596+
'name': {'type': 'string', 'allowed': ['flake8', 'pre-commit']}
597+
},
595598
},
596599
},
597600
}

molecule/verifier/base.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424

2525
from molecule import util
2626
from molecule.verifier.lint import flake8
27+
from molecule.verifier.lint import precommit
2728
from molecule.verifier.lint import rubocop
2829
from molecule.verifier.lint import yamllint
2930
from molecule.verifier.lint import ansible_lint
@@ -106,6 +107,8 @@ def lint(self):
106107
lint_name = self._config.config['verifier']['lint']['name']
107108
if lint_name == 'flake8':
108109
return flake8.Flake8(self._config)
110+
if lint_name == 'pre-commit':
111+
return precommit.PreCommit(self._config)
109112
if lint_name == 'rubocop':
110113
return rubocop.RuboCop(self._config)
111114
if lint_name == 'yamllint':
Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# Copyright (c) 2019 Red Hat, Inc.
2+
#
3+
# Permission is hereby granted, free of charge, to any person obtaining a copy
4+
# of this software and associated documentation files (the "Software"), to
5+
# deal in the Software without restriction, including without limitation the
6+
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
7+
# sell copies of the Software, and to permit persons to whom the Software is
8+
# furnished to do so, subject to the following conditions:
9+
#
10+
# The above copyright notice and this permission notice shall be included in
11+
# all copies or substantial portions of the Software.
12+
#
13+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
18+
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
19+
# DEALINGS IN THE SOFTWARE.
20+
"""Linter module for pre-commit related code."""
21+
22+
import os
23+
24+
import sh
25+
26+
from molecule import logger
27+
from molecule import util
28+
from molecule.verifier.lint import base
29+
30+
LOG = logger.get_logger(__name__)
31+
32+
33+
class PreCommit(base.Base):
34+
"""
35+
Pre-commit tool verifier wrapper.
36+
37+
This class is used to lint files by executing the pre-commit
38+
command line tool for files in the test folder with a prefix
39+
of ``test_``.
40+
41+
`Pre-Commit`_ is not the default verifier linter.
42+
43+
`Pre-Commit`_ is a linter for python files and more.
44+
45+
Additional options can be passed to ``pre-commit`` through the options
46+
dict. Any option set in this section will override the defaults.
47+
48+
.. code-block:: yaml
49+
50+
verifier:
51+
name: testinfra
52+
lint:
53+
name: pre-commit
54+
options:
55+
remove-tabs:
56+
57+
Test file linting can be disabled by setting ``enabled`` to False.
58+
59+
.. code-block:: yaml
60+
61+
verifier:
62+
name: testinfra
63+
lint:
64+
name: pre-commit
65+
enabled: False
66+
67+
Environment variables can be passed to lint.
68+
69+
.. code-block:: yaml
70+
71+
verifier:
72+
name: testinfra
73+
lint:
74+
name: pre-commit
75+
env:
76+
FOO: bar
77+
78+
Example pre-commit configuration file (``.pre-commit-config.yaml``) to run
79+
flake8.
80+
81+
.. code-block:: yaml
82+
83+
repos:
84+
- repo: local
85+
hooks:
86+
- id: flake8
87+
name: flake8
88+
entry: python -m flake8 --max-line-length=120
89+
language: system
90+
types: [python]
91+
92+
.. _`Pre-Commit`: https://pre-commit.com/
93+
"""
94+
95+
def __init__(self, config):
96+
"""
97+
Set up the requirements to execute ``pre-commit`` tool.
98+
99+
:param config: An instance of a Molecule config.
100+
"""
101+
super(PreCommit, self).__init__(config)
102+
self._precommit_command = None
103+
if config:
104+
self._tests = self._get_tests()
105+
106+
@property
107+
def default_options(self):
108+
"""Default options for pre-commit tool runtime."""
109+
return {}
110+
111+
@property
112+
def default_env(self):
113+
"""Default environment variables for pre-commit tool runtime."""
114+
return util.merge_dicts(os.environ.copy(), self._config.env)
115+
116+
def bake(self):
117+
"""Bake a ready to execute ``pre-commit`` command."""
118+
self._precommit_command = sh.Command('pre-commit').bake(
119+
'run',
120+
self.options,
121+
'--files',
122+
self._tests,
123+
_env=self.env,
124+
_out=LOG.out,
125+
_err=LOG.error,
126+
)
127+
128+
def execute(self):
129+
"""Execute the pre-commit command."""
130+
if not self.enabled:
131+
msg = 'Skipping, verifier_lint is disabled.'
132+
LOG.warn(msg)
133+
return
134+
135+
if not self._tests:
136+
msg = 'Skipping, no tests found.'
137+
LOG.warn(msg)
138+
return
139+
140+
if self._precommit_command is None:
141+
self.bake()
142+
143+
msg = 'Executing pre-commit on files found in {}/...'.format(
144+
self._config.verifier.directory
145+
)
146+
LOG.info(msg)
147+
148+
try:
149+
util.run_command(self._precommit_command, debug=self._config.debug)
150+
msg = 'Lint completed successfully.'
151+
LOG.success(msg)
152+
except sh.ErrorReturnCode as e:
153+
util.sysexit(e.exit_code)
154+
155+
def _get_tests(self):
156+
"""
157+
Get a list of test files from the verifier's directory.
158+
159+
:return: list
160+
"""
161+
return list(util.os_walk(self._config.verifier.directory, 'test_*.py'))

setup.cfg

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ install_requires =
8282
paramiko >= 2.5.0, < 3
8383
pathlib2; python_version<"3.2"
8484
pexpect >= 4.6.0, < 5
85+
pre-commit >= 1.17.0, < 2
8586
psutil >= 5.4.6, < 6; sys_platform!="win32" and sys_platform!="cygwin"
8687
PyYAML >= 5.1, < 6
8788
sh >= 1.12.14

test/functional/docker/test_scenarios.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -383,6 +383,27 @@ def test_interpolation(scenario_to_test, with_scenario, scenario_name):
383383
pytest.helpers.run_command(cmd, env=env)
384384

385385

386+
@pytest.mark.parametrize(
387+
'scenario_to_test, driver_name, scenario_name',
388+
[('verifier', 'docker', 'testinfra-pre-commit')],
389+
indirect=['scenario_to_test', 'driver_name', 'scenario_name'],
390+
)
391+
def test_command_verify_testinfra_precommit(
392+
scenario_to_test, with_scenario, scenario_name
393+
):
394+
options = {'scenario_name': scenario_name}
395+
cmd = sh.molecule.bake('create', **options)
396+
pytest.helpers.run_command(cmd)
397+
398+
options = {'scenario_name': scenario_name}
399+
cmd = sh.molecule.bake('converge', **options)
400+
pytest.helpers.run_command(cmd)
401+
402+
options = {'scenario_name': scenario_name}
403+
cmd = sh.molecule.bake('verify', **options)
404+
pytest.helpers.run_command(cmd)
405+
406+
386407
@pytest.mark.parametrize(
387408
'scenario_to_test, driver_name, scenario_name',
388409
[('verifier', 'docker', 'testinfra')],
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
repos:
3+
- repo: local
4+
hooks:
5+
- id: flake8
6+
name: flake8
7+
entry: python -m flake8 --max-line-length=120
8+
language: system
9+
types: [python]
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
dependency:
3+
name: galaxy
4+
driver:
5+
name: docker
6+
lint:
7+
name: yamllint
8+
options:
9+
config-file: ../../resources/.yamllint
10+
platforms:
11+
- name: instance
12+
image: centos:latest
13+
provisioner:
14+
name: ansible
15+
playbooks:
16+
create: ../../../../resources/playbooks/docker/create.yml
17+
destroy: ../../../../resources/playbooks/docker/destroy.yml
18+
lint:
19+
name: ansible-lint
20+
scenario:
21+
name: testinfra-pre-commit
22+
verifier:
23+
name: testinfra
24+
options:
25+
vvv: true
26+
additional_files_or_dirs:
27+
- ../shared/test_*.py
28+
lint:
29+
name: pre-commit
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
- name: Converge
3+
hosts: all
4+
tasks:
5+
- name: Create /tmp/molecule
6+
file:
7+
dest: /etc/molecule
8+
group: root
9+
owner: root
10+
mode: 0755
11+
state: directory
12+
13+
- name: Create /etc/molecule/{{ ansible_hostname }}
14+
copy:
15+
dest: "/etc/molecule/{{ ansible_hostname }}"
16+
group: root
17+
owner: root
18+
mode: 0644
19+
content: "{{ ansible_hostname }}"
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/usr/bin/python
2+
# -*- coding: utf-8 -*-
3+
4+
5+
def test_ansible_hostname(host):
6+
f = host.file('/tmp/molecule/instance-1')
7+
assert not f.exists

0 commit comments

Comments
 (0)