issue #160 - fix doctrine/orm joined inheritance anonymization (#164)

* issue #160 - fix doctrine/orm joined inheritance anonymization

Author: pounard

CHANGELOG.md

@@ -3,12 +3,16 @@
 ## Next
 
 * [feature] ⭐️ Add Doctrine DBAL 4.0 compatibility (#140).
+* [feature] ⭐️ Add Doctrine ORM 3.0 compatibility as a side effect of Doctrine DBAL 4.0 support (#140).
 * [feature] ⭐️ Anonymization - Add Doctrine Embeddables support (#105).
-* [feature] ⭐️ As a side effect, Doctrine ORM 3.0 should now work (#140).
+* [feature] ⭐️ Anonymization - Add Doctrine entity joined inheritance support (#160)
+* [feature] ⭐️ Anonymization - Finalized and improved IBAN/BIC anonymizer (#4)
+* [fix] Restored MySQL 5.7 support (#124)
 * [internal] Remove `doctrine/dbal` dependency from all code except the database session registry (#142).
 * [internal] Introduce `DatabaseSessionRegistry` as single entry point for plugging-in database (#142).
 * [internal] Use `makinacorpus/query-builder` schema manager for DDL alteration (#140).
 * [internal] Raise `makinacorpus/query-builder` dependency to version 1.5.5 (#140).
+* [internal] Many improvements in local/CI `./dev.sh` test script.
 
 ## 1.1.0
 

docs/.vitepress/config.ts

@@ -73,6 +73,7 @@ export default defineConfig({
           { text: 'Custom Anonymizers', link: '/anonymization/custom-anonymizers' },
           { text: 'Anonymization command', link: '/anonymization/command' },
           { text: 'GDPR-friendly workflow', link: '/anonymization/workflow' },
+          { text: 'Doctrine and inheritance', link: '/anonymization/doctrine-inheritance' },
           { text: 'Performance', link: '/anonymization/performance' },
           { text: 'Internals', link: '/anonymization/internals' },
         ]

docs/content/anonymization/custom-anonymizers.md

@@ -74,9 +74,7 @@ use MakinaCorpus\DbToolsBundle\Attribute\AsAnonymizer;
 class MyEnumAnonymizer extends AbstractEnumAnonymizer
 {
 
-    /**
-     * {@inheritdoc}
-     */
+    #[\Override]
     protected function getSample(): array
     {
         // Generate here your sample.
@@ -108,9 +106,7 @@ use MakinaCorpus\DbToolsBundle\Attribute\AsAnonymizer;
 )]
 class MyMulticolumnAnonymizer extends AbstractMultipleColumnAnonymizer
 {
-    /**
-     * @inheritdoc
-     */
+    #[\Override]
     protected function getColumnNames(): array
     {
         // Declare here name fo each part of your multicolumn
@@ -122,9 +118,7 @@ class MyMulticolumnAnonymizer extends AbstractMultipleColumnAnonymizer
         ];
     }
 
-    /**
-     * {@inheritdoc}
-     */
+    #[\Override]
     protected function getSample(): array
     {
         // Generate here your sample.

docs/content/anonymization/doctrine-inheritance.md

@@ -0,0 +1,50 @@
+# Doctrine ORM and entity inheritance
+
+Doctrine ORM allows various complex inheritance scenarios, support of those scenarios
+is usable in some case, but partial or non working in others.
+
+## Embeddables
+
+Embeddables are not quite inheritance, but composition instead. This use case is
+fully supported since all properties from the nested entity live in the entity
+that uses it table.
+
+When the anonymizators builds the SQL query for anonymization, all columns are
+in the same table and run all at once.
+
+## Joined inheritance
+
+Joined inheritance has a very basic testing scenario, and should work flawlessly
+in most cases. When doing joined inheritance, the parent entity table exists
+separatly from the concrete entity implementation.
+
+Anonymizator will hence build up two different SQL queries for anonymizing and
+live with it, one for the child class, and the other for the parent class.
+
+Yes, it has a few drawbacks explained below.
+
+:::warning
+**You cannot use a multi-column anonymizator on columns from the parent entity and columns of the child entity at the same time.**
+
+By design, this API prevents this, and there will never be any work around.
+:::
+
+:::warning
+**When anonymizing parent entity columns, the SQL query will not restrict to the a certain child type.**
+
+This means that all entities in the discriminator map will be anonymized at once,
+none will be filtered out.
+:::
+
+## Other inheritance types
+
+Other inheritances types have not be tested yet, but may work.
+
+:::tip
+The underlaying code handling inheritance in the anonymizator attribute lookup
+is generic enough so other inheritance types may actually work gracefully and
+transparently.
+
+Testing it into your project is easy, [reporting an issue](https://github.com/makinacorpus/DbToolsBundle/issues)
+is as well! We are here to help you whenever a problem arise.
+:::

docs/content/anonymization/essentials.md

@@ -21,6 +21,11 @@ If Doctrine ORM is enabled, the *DbToolsBundle* will automatically look for attr
 If you want to use YAML configuration, look at the [Bundle Configuration
 section](../configuration#anonymization) to see how to configure it.
 
+:::info
+All anonymizers can be configured via attributes on Doctrine ORM entities, but inheritance
+is not fully supported yet, [please read this page](doctrine-inheritance) for more information.
+:::
+
 The anonymization is based on *Anonymizers*. An *Anonymizer* represents a way to anonymize a column (or
 multiple columns in certain cases). For example, you will use the EmailAnonymizer to anonymize a column that
 represents an email address.

docs/content/anonymization/internals.md

@@ -8,14 +8,14 @@ Each anonymizer adds its own `SET` statements to the query, and a few `JOIN` cla
 
 ## PHP query builder
 
-Generating this kind of queries is quiet complexe and mostly impossible without a complete
+Generating this kind of queries is quite complex and mostly impossible without a complete
 and robust SQL query builder.
 
 Our first thought was to use the [DBAL query builder](https://www.doctrine-project.org/projects/doctrine-dbal/en/4.0/reference/query-builder.html#sql-query-builder).
 But, while it is very robust, it lacks many features. Features such as
 update with join, which are essential for our use case.
 
-Instead, we decided to re-use one of our tool: the [makinacorpus/query-builder-bundle](https://github.com/makinacorpus/query-builder-bundle) package.
+Instead, we decided to re-use one of our tool: [makinacorpus/php-query-builder](https://github.com/makinacorpus/query-builder-bundle).
 
 This query builder lets you write SQL queries using a concise and easy to read fluent PHP API
 and [implements a lot of features](https://php-query-builder.readthedocs.io/en/stable/introduction/features.html).

src/Anonymization/Config/Loader/AttributesLoader.php

@@ -28,7 +28,8 @@ public function load(AnonymizationConfig $config): void
             return;
         }
 
-        $metadatas = $entityManager->getMetadataFactory()->getAllMetadata();
+        $metadataFactory = $entityManager->getMetadataFactory();
+        $metadatas = $metadataFactory->getAllMetadata();
 
         foreach ($metadatas as $metadata) {
             \assert($metadata instanceof ClassMetadata);
@@ -51,11 +52,13 @@ public function load(AnonymizationConfig $config): void
 
             $reflexionClass = $metadata->getReflectionClass();
             if ($attributes = $reflexionClass->getAttributes(Anonymize::class)) {
+                // There can only be one of those attributes, foreach() is
+                // required because of reflection API signature.
                 foreach ($attributes as $key => $attribute) {
                     $anonymization = $attribute->newInstance();
                     $config->add(new AnonymizerConfig(
                         $metadata->getTableName(),
-                        // For a anonymization setted on table, we give an arbitrary name
+                        // For a anonymization setted on table, we give an arbitrary name.
                         $anonymization->type . '_' . $key,
                         $anonymization->type,
                         new Options($anonymization->options),
@@ -64,11 +67,11 @@ public function load(AnonymizationConfig $config): void
             }
 
             foreach ($metadata->fieldMappings as $fieldName => $fieldValues) {
-                // Field name with dot are part of Embeddables
+                // Field name with dot are part of Embeddables.
                 if (\str_contains($fieldName, '.')) {
                     if (\key_exists($fieldValues['originalClass'], $embeddedClassesConfig)) {
                         $embeddedClassConfig = $embeddedClassesConfig[$fieldValues['originalClass']];
-                        if(\key_exists($fieldValues['originalField'], $embeddedClassConfig)) {
+                        if (\key_exists($fieldValues['originalField'], $embeddedClassConfig)) {
                             $propertyConfig = $embeddedClassConfig[$fieldValues['originalField']];
                             $config->add(new AnonymizerConfig(
                                 $metadata->getTableName(),
@@ -81,12 +84,29 @@ public function load(AnonymizationConfig $config): void
                     continue;
                 }
 
-                $reflexionProperty = $reflexionClass->getProperty($fieldName);
-                if ($attributes = $reflexionProperty->getAttributes(Anonymize::class)) {
+                $columnName = $metadata->getColumnName($fieldName);
+                if ($metadata->isInheritedField($fieldName)) {
+                    $fieldMapping = $metadata->getFieldMapping($fieldName);
+                    // @phpstan-ignore-next-line
+                    if (\is_array($fieldMapping)) {
+                        // Code for doctrine/orm:^2.0.
+                        $ownerClass = $fieldMapping['inherited'];
+                    } else {
+                        // Code for doctrine/orm:^3.0.
+                        $ownerClass = $fieldMapping->inherited;
+                    }
+                    $parentMetadata = $metadataFactory->getMetadataFor($ownerClass);
+                    \assert($parentMetadata instanceof ClassMetadata);
+                    $tableName = $parentMetadata->getTableName();
+                } else {
+                    $tableName = $metadata->getTableName();
+                }
+
+                if ($attributes = $metadata->getReflectionProperty($fieldName)->getAttributes(Anonymize::class)) {
                     $anonymization = $attributes[0]->newInstance();
                     $config->add(new AnonymizerConfig(
-                        $metadata->getTableName(),
-                        $metadata->getColumnName($fieldName),
+                        $tableName,
+                        $columnName,
                         $anonymization->type,
                         new Options($anonymization->options),
                     ));

tests/Resources/Loader/TestJoinedChild.php

@@ -0,0 +1,31 @@
+<?php
+
+declare(strict_types=1);
+
+namespace MakinaCorpus\DbToolsBundle\Tests\Resources\Loader;
+
+use Doctrine\ORM\Mapping as ORM;
+use MakinaCorpus\DbToolsBundle\Attribute\Anonymize;
+
+#[ORM\Entity()]
+#[ORM\Table(name: 'test_joined_child')]
+class TestJoinedChild extends TestJoinedParent
+{
+    #[Anonymize(type: 'constant', options: ['value' => 'https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Accueil_principal#/media/Fichier:Myotis_crypticus_-_Manuel_Ruedi.jpg'])]
+    #[ORM\Column(length: 255, nullable: true)]
+    private ?string $url = null;
+
+    #[Anonymize(type: 'constant', options: ['value' => 'https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Accueil_principal#/media/Fichier:Myotis_crypticus_-_Manuel_Ruedi.jpg'])]
+    #[ORM\Column(length: 255, nullable: true)]
+    private ?string $thumbnail_url = null;
+
+    public function getUrl(): ?string
+    {
+        return $this->url;
+    }
+
+    public function getThumbnailUrl(): ?string
+    {
+        return $this->thumbnail_url;
+    }
+}

tests/Resources/Loader/TestJoinedParent.php

@@ -0,0 +1,37 @@
+<?php
+
+declare(strict_types=1);
+
+namespace MakinaCorpus\DbToolsBundle\Tests\Resources\Loader;
+
+use Doctrine\ORM\Mapping as ORM;
+use MakinaCorpus\DbToolsBundle\Attribute\Anonymize;
+
+#[ORM\Entity()]
+#[ORM\Table(name: 'test_joined_parent')]
+#[ORM\InheritanceType('JOINED')]
+#[ORM\DiscriminatorColumn(name: 'type', type: 'string')]
+#[ORM\DiscriminatorMap([
+    'child' => TestJoinedChild::class,
+])]
+class TestJoinedParent
+{
+    #[ORM\Id]
+    #[ORM\GeneratedValue(strategy: "NONE")]
+    #[ORM\Column]
+    private ?int $id = null;
+
+    #[ORM\Column(length: 180, unique: true)]
+    #[Anonymize(type:'email', options: ['domain' => 'toto.com'])]
+    private ?string $email = null;
+
+    public function getId(): ?int
+    {
+        return $this->id;
+    }
+
+    public function getEmail(): ?string
+    {
+        return $this->email;
+    }
+}

tests/Unit/Anonymization/Configuration/AttributeLoaderTest.php

@@ -12,10 +12,10 @@
 use MakinaCorpus\DbToolsBundle\Anonymization\Config\AnonymizationConfig;
 use MakinaCorpus\DbToolsBundle\Anonymization\Config\AnonymizerConfig;
 use MakinaCorpus\DbToolsBundle\Anonymization\Config\Loader\AttributesLoader;
-use MakinaCorpus\DbToolsBundle\Tests\Resources\Loader\TestEntity;
 use MakinaCorpus\DbToolsBundle\Test\UnitTestCase;
-use MakinaCorpus\DbToolsBundle\Tests\Resources\Loader\TestEmbeddableEntity;
+use MakinaCorpus\DbToolsBundle\Tests\Resources\Loader\TestEntity;
 use MakinaCorpus\DbToolsBundle\Tests\Resources\Loader\TestEntityWithEmbedded;
+use MakinaCorpus\DbToolsBundle\Tests\Resources\Loader\TestJoinedChild;
 
 class AttributeLoaderTest extends UnitTestCase
 {
@@ -68,6 +68,43 @@ public function testLoadOk(): void
         self::assertSame('country', $testTableConfig['address_0']->options->get('country'));
     }
 
+    public function testLoadWithJoinedInheritance(): void
+    {
+        $classMetadataFactory = $this->getClassMetadataFactory();
+        $classMetadataFactory->getMetadataFor(TestJoinedChild::class);
+
+        $entityManager = $this->createMock(EntityManagerInterface::class);
+        $entityManager
+            ->expects($this->exactly(1))
+            ->method('getMetadataFactory')
+            ->willReturn($classMetadataFactory)
+        ;
+
+        $entityManagerProvider = $this->createMock(EntityManagerProvider::class);
+        $entityManagerProvider
+            ->expects($this->exactly(1))
+            ->method('getManager')
+            ->willReturn($entityManager)
+        ;
+
+        // We load configuration for the 'default' connection.
+        $config = new AnonymizationConfig('default');
+        (new AttributesLoader($entityManagerProvider))->load($config);
+
+        // Then we validate what's in it:
+        $testTableConfig = $config->getTableConfig('test_joined_child');
+        self::assertCount(2, $testTableConfig);
+        self::assertInstanceOf(AnonymizerConfig::class, $testTableConfig['url']);
+        self::assertSame('constant', $testTableConfig['url']->anonymizer);
+        self::assertInstanceOf(AnonymizerConfig::class, $testTableConfig['thumbnail_url']);
+        self::assertSame('constant', $testTableConfig['thumbnail_url']->anonymizer);
+
+        $testTableConfig = $config->getTableConfig('test_joined_parent');
+        self::assertCount(1, $testTableConfig);
+        self::assertInstanceOf(AnonymizerConfig::class, $testTableConfig['email']);
+        self::assertSame('email', $testTableConfig['email']->anonymizer);
+    }
+
     public function testLoadWithEmbeddedOk(): void
     {
         $classMetadataFactory = $this->getClassMetadataFactory();
@@ -87,7 +124,7 @@ public function testLoadWithEmbeddedOk(): void
             ->willReturn($entityManager)
         ;
 
-        // We try to load configuration for the 'default' connection.
+        // We load configuration for the 'default' connection.
         $config = new AnonymizationConfig('default');
         (new AttributesLoader($entityManagerProvider))->load($config);