Merge branch 'release' into 'master'

v4.4.1 See merge request passbolt/passbolt-ce-api!207
passbolt · Nov 21, 2023 · 9ee0842 · 9ee0842
2 parents eb064b9 + 04b9c6d
commit 9ee0842
Show file tree

Hide file tree

Showing 72 changed files with 2,068 additions and 2,359 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,36 @@
 All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).
 
+## [4.4.1] - 2023-11-20
+### Improved
+- PB-28521 Alter the gpgkeys.uid column length to 769 to enable the synchronisation of user with very long names
+
+### Fixed
+- PB-27957 As an administrator I should be notified that an administrator was deleted only when an administrator has been deleted, and not a regular user
+
+### Maintenance
+- PB-27927 Remove unused user_agents table
+- PB-28616 Refactor the email digest plugin code for easier usage and maintainability
+
+## [4.4.1-test.3] - 2023-11-20
+### Fixed
+- PB-27616 Revert performance improvements as further investigation is required
+
+## [4.4.1-test.2] - 2023-11-20
+### Maintenance
+- PB-28521 Alter the gpgkeys.uid column length to 769
+
+## [4.4.1-test.1] - 2023-11-17
+### Improved
+- PB-27616 As a user I should see improved performances when retrieving resources on the GET resources.json entry point
+
+### Fixed
+- PB-27957 As an administrator I should be notified that an administrator was deleted only when an administrator has been deleted, and not a regular user
+
+### Maintenance
+- PB-27927 The unused user_agents table is now removed from the database
+- PB-28616 The email digest plugin code was deeply refactored for enhanced usage and maintainability
+
 ## [4.4.0] - 2023-11-07
 ### Added
 - PB-27773 As an administrator I can deny access to the mobile setup screen with RBAC

diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
@@ -1,37 +1,18 @@
-Release song: https://www.youtube.com/watch?v=6Ejga4kJUts
+Release song: https://youtu.be/RbmS3tQJ7Os?si=lp8QM5B-R65C8Jek
 
-Version 4.4 of the Community Edition has launched with new capabilities and improvements.
+Passbolt v4.4.1 is a maintenance release aimed at addressing issues reported by the community, which were introduced in the previous release.
 
-With this release, users are able to manage TOTPs directly from the browser, providing an extended TOTP experience across all their devices. They can now be created, deleted, organised and shared with others just like any other resource type.
+The update addresses an issue concerning user roles in email notifications. Previously, administrators received notifications when another administrator was deleted. However, the deletion of any user, regardless of their administrator status, was incorrectly reported as an administrator deletion. This issue has been resolved.
 
-Another highlight of this release, administrators now have the ability to suspend/unsuspend users. This new feature will offer administrators with more control over access management of their instance. By example, they will be able to prevent access to the passbolt instance for users in temporary leave, therefore enforce company policies.
-
-And that's not all – a number of fixes and enhancements have been implemented to improve user experience. Among them, notification emails are now aggregated in certain cases, including limiting emails when a user imports a large amount of passwords.
-
-If you’re a system operator, please note that using older PHP versions will now trigger a healthcheck warning. Support for PHP 7.4 and 8.0 will be discontinued soon. Admins are encouraged to upgrade to PHP 8.1 or higher and use the latest version of the passbolt API.
-
-Get the most out of passbolt – upgrade to version 4.4. Thanks for continuing to support passbolt and for being part of the community!
-
-
-## [4.4.0] - 2023-11-07
-### Added
-- PB-27773 As an administrator I can deny access to the mobile setup screen with RBAC
-- PB-27951 As system operator I should be warned in the healthcheck when using PHP < 8.1, as support for PHP versions 7.4 and 8.0 will soon be removed
+We extend our gratitude to the community members who reported these issues. Your support and patience are greatly appreciated.
 
+## [4.4.1] - 2023-11-20
 ### Improved
-- PB-27948 Guest identification by their username should be case-insensitive, unless specified in configuration
-- PB-27957 Send notifications to all administrators when an administrator is deleted
-- PB-27941 Send notifications to administrators when an administrator loses its administrator role
-- PB-28171 Enable the email digest by default
-
-### Security
-- PB-28274 Fixes an XSS Security issue with mail content sanitization
+- PB-28521 Alter the gpgkeys.uid column length to 769 to enable the synchronisation of user with very long names
 
 ### Fixed
-- PB-25477 As an administrator, I should be able to recreate a user with an email that exists in the db via the command line
-- PB-27799 As an administrator installing passbolt on PostgreSQL, the database encoding should be set to utf-8
-- PB-27857 Fix help site release notes automation by adding flavour on help site release notes merge request
+- PB-27957 As an administrator I should be notified that an administrator was deleted only when an administrator has been deleted, and not a regular user
 
 ### Maintenance
-- PB-27932 Improve code static by using cakedccakephp/phpstan
-- PB-28079 Remove deprecation warnings from the test suite
+- PB-27927 Remove unused user_agents table
+- PB-28616 Refactor the email digest plugin code for easier usage and maintainability
diff --git a/config/Migrations/20211121231300_V340MigrateASCIIFieldsEncoding.php b/config/Migrations/20211121231300_V340MigrateASCIIFieldsEncoding.php
@@ -599,15 +599,17 @@ public function up()
             ])
             ->save();
 
-        $this->table('user_agents')
-            ->changeColumn('id', 'uuid', [
-                'default' => null,
-                'limit' => null,
-                'null' => false,
-                'encoding' => 'ascii',
-                'collation' => 'ascii_general_ci'
-            ])
-            ->save();
+        if ($this->hasTable('user_agents')) {
+            $this->table('user_agents')
+                ->changeColumn('id', 'uuid', [
+                    'default' => null,
+                    'limit' => null,
+                    'null' => false,
+                    'encoding' => 'ascii',
+                    'collation' => 'ascii_general_ci'
+                ])
+                ->save();
+        }
 
         $this->table('users')
             ->changeColumn('id', 'uuid', [

diff --git a/config/Migrations/20231108114414_V441AlterUidOnGpgkeys.php b/config/Migrations/20231108114414_V441AlterUidOnGpgkeys.php
@@ -0,0 +1,37 @@
+<?php
+declare(strict_types=1);
+
+/**
+ * Passbolt ~ Open source password manager for teams
+ * Copyright (c) Passbolt SA (https://www.passbolt.com)
+ *
+ * Licensed under GNU Affero General Public License version 3 of the or any later version.
+ * For full copyright and license information, please see the LICENSE.txt
+ * Redistributions of files must retain the above copyright notice.
+ *
+ * @copyright     Copyright (c) Passbolt SA (https://www.passbolt.com)
+ * @license       https://opensource.org/licenses/AGPL-3.0 AGPL License
+ * @link          https://www.passbolt.com Passbolt(tm)
+ * @since         4.4.1
+ */
+
+use Migrations\AbstractMigration;
+
+class V441AlterUidOnGpgkeys extends AbstractMigration
+{
+    /**
+     * Change Method.
+     *
+     * More information on this method is available here:
+     * https://book.cakephp.org/phinx/0/en/migrations.html#the-change-method
+     *
+     * @return void
+     */
+    public function change(): void
+    {
+        $table = $this->table('gpgkeys');
+        $table
+            ->changeColumn('uid', 'string', ['limit' => 769])
+            ->update();
+    }
+}
diff --git a/config/Migrations/20231115235026_V441DropUserAgents.php b/config/Migrations/20231115235026_V441DropUserAgents.php
@@ -0,0 +1,33 @@
+<?php
+declare(strict_types=1);
+
+use Migrations\AbstractMigration;
+
+/**
+ * Passbolt ~ Open source password manager for teams
+ * Copyright (c) Passbolt SA (https://www.passbolt.com)
+ *
+ * Licensed under GNU Affero General Public License version 3 of the or any later version.
+ * For full copyright and license information, please see the LICENSE.txt
+ * Redistributions of files must retain the above copyright notice.
+ *
+ * @copyright     Copyright (c) Passbolt SA (https://www.passbolt.com)
+ * @license       https://opensource.org/licenses/AGPL-3.0 AGPL License
+ * @link          https://www.passbolt.com Passbolt(tm)
+ * @since         4.4.1
+ */
+class V441DropUserAgents extends AbstractMigration
+{
+    /**
+     * Change Method.
+     *
+     * More information on this method is available here:
+     * https://book.cakephp.org/phinx/0/en/migrations.html#the-change-method
+     * @return void
+     */
+    public function change(): void
+    {
+        $table = $this->table('user_agents');
+        $table->drop()->save();
+    }
+}
diff --git a/config/version.php b/config/version.php
@@ -1,8 +1,8 @@
 <?php
 return [
     'passbolt' => [
-        'version' => '4.4.0',
-        'name' => 'Zombie',
+        'version' => '4.4.1',
+        'name' => 'Gimme Shelter',
     ],
     'php' => [
         'minVersion' => '7.4',

diff --git a/phpstan-baseline.neon b/phpstan-baseline.neon
@@ -5,16 +5,6 @@ parameters:
 			count: 1
 			path: plugins/PassboltCe/AccountSettings/src/Model/Table/AccountSettingsTable.php
 
-		-
-			message: "#^Unsafe access to private property Passbolt\\\\EmailDigest\\\\Utility\\\\Digest\\\\DigestsPool\\:\\:\\$instance through static\\:\\:\\.$#"
-			count: 4
-			path: plugins/PassboltCe/EmailDigest/src/Utility/Digest/DigestsPool.php
-
-		-
-			message: "#^Unsafe access to private property Passbolt\\\\EmailDigest\\\\Utility\\\\Factory\\\\DigestFactory\\:\\:\\$instance through static\\:\\:\\.$#"
-			count: 4
-			path: plugins/PassboltCe/EmailDigest/src/Utility/Factory/DigestFactory.php
-
 		-
 			message: "#^Unsafe access to private property Passbolt\\\\EmailNotificationSettings\\\\Utility\\\\EmailNotificationSettings\\:\\:\\$configSettingsSource through static\\:\\:\\.$#"
 			count: 4

diff --git a/plugins/PassboltCe/EmailDigest/README.md b/plugins/PassboltCe/EmailDigest/README.md
@@ -0,0 +1,71 @@
+# Introduction
+After importing a large amount of resources, or any action generating a large amount of emails,
+we would like the emails to be grouped by operator and template into one single email resuming
+the action performed.
+
+When the number of emails ina digest is below or equal 10, all the emails are rendered
+within the email digest.
+
+When the number of emails under a digest is above 10 emails, only a summary email is sent.
+
+This document describes the approach taken.
+
+# How to add a digest template
+
+Typically, the digest templates are added in the plugins, e.g. in EmailDigestPlugin for the default email digests,
+using the `DigestTemplateRegistry`:
+
+```php
+DigestTemplateRegistry::getInstance()
+    ->addTemplate(new ResourceChangesDigestTemplate(5))
+    ->addTemplate(new ResourceShareDigestTemplate(1))
+    ...
+```
+The priority of the digest template may be passed in the constructor. Default is 10, the highest priority is 1.
+
+
+
+# Technical specifications
+
+This section describes the various components involved in the sending of email digest.
+
+## `DigestTemplateRegistry`
+
+This class gathers all the digest templates. Typically, it is used in the `EmailDigestPlugin::bootstrap();`
+to register which digest templates are available. Note that templates are added in when performing an action
+at the command line only.
+
+## `AbstractDigestTemplate`
+
+A digest template defines the email templates that will be covered by a digest, the priority
+of a digest, the template used by this digest when the number of emails is above 10.
+
+Default priority is 10, and a number below enhances the priority. So priority 1 is higher than priority 2.
+
+You may follow the abstract methods of this abstract template to build a new digest template.
+
+## `DigestsCollection`
+
+When sending or previewing a batch of emails (per default 100, see `PASSBOLT_PLUGINS_EMAIL_DIGEST_BATCH_SIZE_LIMIT`),
+the emails are attributed to a digest, if they are candidates to a digest, or will be
+rendered as single emails. The `DigestsCollection` iterates through all the emails
+to be sent and creates digests resp. assigns to these digests the emails candidate.
+
+## `Digest`
+A digest contains the emails for:
+- a single operator
+- a single recipient
+- a single digest template
+
+The `marshalEmails` method renders the emails within the digest, returning an `EmailDigest`.
+
+## `EmailDigest`
+This class is the conversion of a `Digest` into an object that can be sent as an email. It is the email realization
+of a digest, with all the information needed to create an email to be sent.
+
+# Summary
+The developer registers `AbstractDigestTemplate`s. When emails are sent, these are grouped in `Digest`s of the same
+recipient and operator by the `DigestsCollection`, or left as single emails if they do not find a matching digest template.
+
+When sending the emails, each `Digest` is previewed resp. sent thanks to the `EmailDigest` class.
+
diff --git a/plugins/PassboltCe/EmailDigest/src/Command/PreviewCommand.php b/plugins/PassboltCe/EmailDigest/src/Command/PreviewCommand.php
@@ -22,6 +22,8 @@
 use Cake\Console\ConsoleIo;
 use Cake\Console\ConsoleOptionParser;
 use Cake\Core\Configure;
+use Cake\ORM\TableRegistry;
+use Cake\Utility\Hash;
 use Passbolt\EmailDigest\Service\PreviewEmailBatchService;
 
 class PreviewCommand extends PassboltCommand
@@ -57,10 +59,21 @@ public function buildOptionParser(ConsoleOptionParser $parser): ConsoleOptionPar
      */
     public function execute(Arguments $args, ConsoleIo $io): ?int
     {
-        $emailSenderService = new PreviewEmailBatchService();
-
         $limit = (int)$args->getOption('limit');
-        $previews = $emailSenderService->previewNextEmailsBatch($limit);
+        /** @var \EmailQueue\Model\Table\EmailQueueTable $EmailQueueTable */
+        $EmailQueueTable = TableRegistry::getTableLocator()->get('EmailQueue.EmailQueue');
+        $emailQueues = $EmailQueueTable->getBatch($limit);
+
+        Configure::write('App.baseUrl', '/');
+
+        if (!empty($emailQueues)) {
+            // we release the locks as soon as we get the emails
+            // we don't want to block the next batch ran by a cron job because of lock.
+            // technically, to do better, we should write the same query ran in getBatch method without locking the emails
+            $EmailQueueTable->releaseLocks(Hash::extract($emailQueues, '{n}.id'));
+        }
+
+        $previews = (new PreviewEmailBatchService())->previewNextEmailsBatch($emailQueues);
         foreach ($previews as $preview) {
             $io->out($preview->getHeaders());
             if ($args->getOption('body') === true) {

diff --git a/plugins/PassboltCe/EmailDigest/src/Command/SenderCommand.php b/plugins/PassboltCe/EmailDigest/src/Command/SenderCommand.php
@@ -22,6 +22,7 @@
 use Cake\Console\ConsoleIo;
 use Cake\Console\ConsoleOptionParser;
 use Cake\Core\Configure;
+use Cake\ORM\TableRegistry;
 use Passbolt\EmailDigest\Service\SendEmailBatchService;
 
 class SenderCommand extends PassboltCommand
@@ -47,9 +48,11 @@ public function buildOptionParser(ConsoleOptionParser $parser): ConsoleOptionPar
      */
     public function execute(Arguments $args, ConsoleIo $io): ?int
     {
-        $emailSenderService = new SendEmailBatchService();
-
-        $emailSenderService->sendNextEmailsBatch((int)$args->getOption('limit'));
+        $limit = (int)$args->getOption('limit');
+        /** @var \EmailQueue\Model\Table\EmailQueueTable $EmailQueueTable */
+        $EmailQueueTable = TableRegistry::getTableLocator()->get('EmailQueue.EmailQueue');
+        $emails = $EmailQueueTable->getBatch($limit);
+        (new SendEmailBatchService())->sendNextEmailsBatch($emails);
 
         return $this->successCode();
     }

diff --git a/plugins/PassboltCe/EmailDigest/src/EmailDigestPlugin.php b/plugins/PassboltCe/EmailDigest/src/EmailDigestPlugin.php
@@ -16,14 +16,16 @@
  */
 namespace Passbolt\EmailDigest;
 
-use App\Notification\EmailDigest\DigestRegister\GroupDigests;
-use App\Notification\EmailDigest\DigestRegister\ResourceDigests;
+use App\Notification\DigestTemplate\GroupMembershipDigestTemplate;
+use App\Notification\DigestTemplate\GroupUserDeleteDigestTemplate;
+use App\Notification\DigestTemplate\ResourceChangesDigestTemplate;
+use App\Notification\DigestTemplate\ResourceShareDigestTemplate;
 use Cake\Console\CommandCollection;
 use Cake\Core\BasePlugin;
-use Cake\Core\Configure;
 use Cake\Core\PluginApplicationInterface;
 use Passbolt\EmailDigest\Command\PreviewCommand;
 use Passbolt\EmailDigest\Command\SenderCommand;
+use Passbolt\EmailDigest\Utility\Digest\DigestTemplateRegistry;
 
 class EmailDigestPlugin extends BasePlugin
 {
@@ -34,12 +36,12 @@ public function bootstrap(PluginApplicationInterface $app): void
     {
         parent::bootstrap($app);
 
-        if (PHP_SAPI === 'cli' || (Configure::read('debug') && Configure::read('passbolt.selenium.active'))) {
-            // Core email digests
-            $app->getEventManager()
-                ->on(new GroupDigests())
-                ->on(new ResourceDigests());
-        }
+        // Core email digests
+        DigestTemplateRegistry::getInstance()
+            ->addTemplate(new ResourceChangesDigestTemplate())
+            ->addTemplate(new ResourceShareDigestTemplate())
+            ->addTemplate(new GroupUserDeleteDigestTemplate())
+            ->addTemplate(new GroupMembershipDigestTemplate());
     }
 
     /**