Merge branch 'development'

Author: hkage

.github/workflows/python_test.yml .github/workflows/test.yml

@@ -1,6 +1,6 @@
 name: Test
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
 
@@ -11,7 +11,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [2.7, 3.5, 3.6, 3.7, 3.8, 3.9]
+        python-version: ['2.7', '3.6', '3.7', '3.8', '3.9']
 
     steps:
     - uses: actions/checkout@v2

CHANGELOG.md

@@ -2,6 +2,13 @@
 
 ## Development
 
+* [#34](https://github.com/rheinwerk-verlag/postgresql-anonymizer/issues/34): Subprocess "run" being used on Python2.7
+* [#35](https://github.com/rheinwerk-verlag/postgresql-anonymizer/issues/35): parmap no longer supports Python 2.7
+  * Dropped Python 3.5 support
+  * Pinned libraries Python 2.7
+* [#32](https://github.com/rheinwerk-verlag/postgresql-anonymizer/pull/32): Fixed pg_dump arguments ([korsar182](https://github.com/korsar182))
+* Simplified provider registration (no metaclass usage anymore)
+
 ## 0.6.1 (2021-07-13)
 
 * Added missing dependencies for the `setup.py`

CONTRIBUTING.rst

@@ -0,0 +1,60 @@
+Contributing to PostgreSQL Anonymizer
+=====================================
+
+First of all: thanks for your interest in this project and taking the time to contribute.
+
+The following document is a small set of guidelines for contributing to this project. They are guidelines and no rules.
+
+Reporting bugs
+---------------
+
+If you have found a bug, please check the project's `issue`_ page first and feel free to create a `new issue`_, if no
+one else has reported it yet.
+
+Making changes
+--------------
+
+Create a fork if you want to make changes or clone the repo if you want a readonly access to the current development
+version:
+
+.. code-block:: sh
+
+    $ git clone [email protected]:rheinwerk-verlag/postgresql-anonymizer.git
+    $ cd postgresql-anonymizer
+
+We are using `poetry`_ for development, you may need to install it first:
+
+.. code-block:: sh
+
+    $ sudo pip install poetry
+
+Now you can install all development requirements and activate the virtualenv:
+
+.. code-block:: sh
+
+    $ poetry install
+    $ poetry shell
+
+Coding style
+------------
+
+We have created an `EditorConfig`_ file for this project that should be usable for most IDEs. Otherwise please make
+sure to adhere to the specifications from the config file.
+
+Creating a pull request
+-----------------------
+
+Before creating a pull request make sure to check:
+
+* existing docstrings have been updated
+* new code has valid docstrings
+* whether existing `tests`_ have to be fixed
+* new tests have to be written first
+* the documentation (in particular the `Sphinx documentation`_) has to be modified
+
+.. _issue: https://github.com/rheinwerk-verlag/postgresql-anonymizer/issues
+.. _new issue: https://github.com/rheinwerk-verlag/postgresql-anonymizer/issues/new
+.. _poetry: https://python-poetry.org/
+.. _EditorConfig: https://editorconfig.org/
+.. _tests: https://github.com/rheinwerk-verlag/postgresql-anonymizer/tree/development/tests
+.. _Sphinx documentation: https://github.com/rheinwerk-verlag/postgresql-anonymizer/tree/development/docs

README.rst

@@ -11,14 +11,12 @@ It uses a YAML file to define which tables and fields should be anonymized and p
 
 .. contents::
 
-.. section-numbering::
-
 Features
 --------
 
 * Intentionally compatible with Python 2.7 (for old, productive platforms)
-* Anonymize PostgreSQL tables on data level entry with various methods (s. table below)
-* Exclude data for anonymization depending on regular expressions
+* Anonymize PostgreSQL tables on data level entry with various providers (some examples in the table below)
+* Exclude data for anonymization depending on regular expressions or SQL ``WHERE`` clauses
 * Truncate entire tables for unwanted data
 
 +----------------+----------------------+-------------------------+----------------------------------+
@@ -43,22 +41,24 @@ Features
 | ``uuid_col``   | 00010203-0405-...... | ``uuid4``               | f7c1bd87-4d....                  |
 +----------------+----------------------+-------------------------+----------------------------------+
 
-* Note: ``faker.unique.[provider]`` only supported on Python 3.5+ (Faker library min. supported python version)
+* Note: ``faker.unique.[provider]`` only supported on Python 3.6+ (Faker library min. supported python version)
 * Note: ``uuid4`` - only for (native `uuid4`_) columns
 
 See the `documentation`_ for a more detailed description of the provided anonymization methods.
 
 Installation
 ------------
 
-The default installation method is to use ``pip``::
+The default installation method is to use ``pip``:
+
+.. code-block:: sh
 
     $ pip install pganonymize
 
 Usage
 -----
 
-::
+.. code-block:: sh
 
     usage: pganonymize [-h] [-v] [-l] [--schema SCHEMA] [--dbname DBNAME]
                    [--user USER] [--password PASSWORD] [--host HOST]
@@ -87,7 +87,9 @@ all anonymization rules for that database. Take a look at the `schema documentat
 `YAML sample schema`_.
 
 
-Example call::
+Example calls:
+
+.. code-block:: sh
 
     $ pganonymize --schema=myschema.yml \
         --dbname=test_database \
@@ -109,11 +111,15 @@ Database dump
 
 With the ``--dump-file`` argument it is possible to create a dump file after anonymizing the database. Please note,
 that the ``pg_dump`` command from the ``postgresql-client-common`` library is necessary to create the dump file for the
-database, e.g. under Linux::
+database, e.g. under Linux:
+
+.. code-block:: sh
 
-    sudo apt-get install postgresql-client-common
+    $ sudo apt-get install postgresql-client-common
 
-Example call::
+Example call:
+
+.. code-block:: sh
 
     $ pganonymize --schema=myschema.yml \
         --dbname=test_database \
@@ -123,31 +129,18 @@ Example call::
         --dump-file=/tmp/dump.gz \
         -v
 
-Quickstart
-----------
-
-Clone repo::
-
-    $ git clone [email protected]:rheinwerk-verlag/postgresql-anonymizer.git
-    $ cd postgresql-anonymizer
-
-For making changes and developing pganonymizer, you need to install ``poetry``::
-
-    $ sudo pip install poetry
-
-Now you can install all requirements and activate the virtualenv::
-
-    $ poetry install
-    $ poetry shell
-
 Docker
-------
+~~~~~~
+
+If you want to run the anonymizer within a Docker container you first have to build the image:
 
-If you want to run the anonymizer within a Docker container you first have to build the image::
+.. code-block:: sh
 
     $ docker build -t pganonymizer .
 
-After that you can pass a schema file to the container, using Docker volumes, and call the anonymizer::
+After that you can pass a schema file to the container, using Docker volumes, and call the anonymizer:
+
+.. code-block:: sh
 
     $ docker run \
         -v <path to your schema>:/schema.yml \

pganonymizer/cli.py

@@ -9,7 +9,7 @@
 import yaml
 
 from pganonymizer.constants import DATABASE_ARGS, DEFAULT_SCHEMA_FILE
-from pganonymizer.providers import PROVIDERS
+from pganonymizer.providers import provider_registry
 from pganonymizer.utils import anonymize_tables, create_database_dump, get_connection, truncate_tables
 
 
@@ -28,8 +28,8 @@ def get_pg_args(args):
 def list_provider_classes():
     """List all available provider classes."""
     print('Available provider classes:\n')
-    for provider_cls in PROVIDERS:
-        print('{:<10} {}'.format(provider_cls.id, provider_cls.__doc__))
+    for key, provider_cls in provider_registry.providers.items():
+        print('{:<10} {}'.format(key, provider_cls.__doc__))
 
 
 def get_arg_parser():

pganonymizer/exceptions.py

@@ -14,5 +14,9 @@ class InvalidProviderArgument(PgAnonymizeException):
     """Raised if an argument is unknown or invalid for a provider."""
 
 
+class ProviderAlreadyRegistered(PgAnonymizeException):
+    """Raised if another provider with the same id has already been registered."""
+
+
 class BadDataFormat(PgAnonymizeException):
     """Raised if the anonymized data cannot be copied."""