14.0 mass merge lite

mass_merge_lite/README.rst

@@ -0,0 +1,94 @@
+.. image:: https://img.shields.io/badge/license-LGPL--3-blue.png
+   :target: https://www.gnu.org/licenses/lgpl
+   :alt: License: LGPL-3
+
+Mass Merge Records
+==================
+This module is a general purpose module that merges records that may be similar
+or reflect similar properties when created.
+Merges any number of records from any table. Similar to the base_partner_merge functionality,
+but with any table and with an interfaces that permits partial merges.
+
+Considerations
+--------------
+
+Technical Explanation
+---------------------
+For demonstration there can exist a customer with two or more record entries that
+are same apart from difference in names. Among the records there could be caps
+in the name characters. e.g
+** 1.) Paul
+** 2.) PAul
+** 3.) pAuL
+the above records are just one "__Paul__" so in this case you can merge the
+records into one.
+
+Functional Usage
+----------------
+Go to Settings - Manage user - Tick Mass Merge	boolean field to give access.
+
+Go to Settings - Technical - Merge records. There are two tools:
+
+* Record Merge by ID: Given n ids from a table, merge them into a single record
+* Record Merge by criteria: Given a filter/domain on a table and a criteria to group the filtered records, create the corresponding "Merge by ID" models
+
+The Merge by ID is the basic tool, in "Draft" state you have to provide:
+
+* The model
+* N ids to merge and mark one of them as destiny (i.e. the record that the others will be merged upon)
+
+Press Launch to get to the "In progress" state. Several new tabs appear:
+
+* The first one is the Relation Fields, i.e. the fields that have a relationship with other tables (o2m and m2o), and should/could be considered for possible merging. The tree shows the number of records and a button to create a merge for those records (a Merge by Id in case of m2o relationship and a Merge by Criteria in case of o2m relationship).
+* The next four tabs are the steps of the Merge in itself:
+
+ * Data consolidation: Copying the data from the merged records to the destiny one. You can choose which fields to merge and the order of consolidation (i.e. which record data will take prevalence)
+ * FKs merge: Changing all fks in the database pointing to the merged records to the destiny one. The code of the base_parter_merge is used. You can choose which fields to merge.
+ * References merge: Changing all reference fields (e.g. field value = res.partner, 145) to the destiny one. You can choose which fields to merge.
+ * Non-relational merge: Changing all reference fields (two fields, one with model = res.partner, other res_id = 145) to the destiny one. You can choose which fields to merge.
+
+* The last two tabs are the steps to consider after the merge:
+
+ * Recompute: List of computed fields that are stored and could need a recompute. You can choose which fields to recompute.
+ * Delete merged: What to do with the merged records (not the destiny). You can delete (launch unlink), deactivate (write active False), recompute (all compute stored fields are recomputed), SQL delete (warning, just in case the ORM does not let us), or leave them (None).
+
+A Merge Button is provided to launch everything in the correct order, getting to the Done state.
+
+The Merge by Criteria is a process to generate the Merge by ID automatically. In "Draft" state you have to provide:
+
+* The model
+* A filter: A domain to get all records that will be merged
+* A group key: A python expression that will give the key that groups the records resulting of the filter in different merges
+* An order: an optional _order expression to apply in each merge group to determine the destiny one. If none is provided, the table _order is used
+
+Press Start to get to the "In progress" state. Several new tabs appear:
+* Merge groups: A list of all the merges that will be generated, presenting the destiny id. You can choose which groups will be merged
+* The same six tabs that appear in the Merge by ID model, and that will be used as default values for the generated groups (i.e. you can configure all the mergings before they are created)
+
+Press Create merges to generate all the Merge by IDs records and get to the Done state. Two buttons let you launch processes on all the groups:
+* Merge all groups
+* Cancel all groups
+
+Be warned: This module is both dangerous and useful, you can solve big problems in a very small time but you can create bigger problems as easily. Test any merges in a duplicate database before doing anything in production.
+
+Example: There are two projects that should be merged, as they are really the same project.
+
+* We start creating the Merge by ID on the project.project model, providing the ids
+* The relation fields tab warns us that we should consider many models:
+
+ * Project.task: Both projects come from a template, so they each have 15 tasks that start with the same code. We press the button + in the task field lines to create a Merge by criteria. The filter is already filled, so we only need a group key. In this case the task should be merged two by two by the code (6 first letters of the name), so it would be: o.name[:6] . We launch all the merges
+ * Other models: The analytic account / contract, sale order, sale order line...
+
+With this module all this merge is done in minutes.
+
+Also, this module could be inherited to create interfaces for the final user for a certain model (e.g. product merge).
+
+Known issues / Roadmap
+----------------------
+
+- Tests: merge 2, 3 countries; check if partner countries have changed,
+  check if ir_model_data was updated
+- On merge wizard, show a visual link that opens the ref in a pop window
+  (such as web_tree_many2one_clickable offers)
+- Generic support for '_inherits' models: when one is merged, merge the other
+  (now we only have specific support for product.product/product.template)

mass_merge_lite/__init__.py

@@ -0,0 +1,2 @@
+from . import models
+from . import tools

mass_merge_lite/__manifest__.py

@@ -0,0 +1,23 @@
+#  Copyright (C) 2010-2019 Today OpenERP SA (<http://www.openerp.com>)
+#  Sunflower IT (<https://www.sunflowerweb.nl>)
+#  programmed by: Oscar Alcala: [email protected]
+#  programmed by: Jose Morales: [email protected]
+#  programmed by: Sunflower IT: [email protected]
+#  License GNU General Public License see <http://www.gnu.org/licenses/>
+{
+    "name": "Mass Merge Records - lite version",
+    "version": "14.0.1.0.0",
+    "author": "Vauxoo, Odoo Community Association (OCA)",
+    "category": "Tools",
+    "website": "https://github.com/OCA/server-ux",
+    "license": "AGPL-3",
+    "depends": ["base"],
+    "data": [
+        "security/merge_security.xml",
+        "security/ir.model.access.csv",
+        "views/record_merge_id.xml",
+        "views/record_merge_criteria.xml",
+    ],
+    "installable": True,
+    "application": True,
+}

mass_merge_lite/models/__init__.py

@@ -0,0 +1,3 @@
+from . import record_merge_mixin
+from . import record_merge_id
+from . import record_merge_criteria

mass_merge_lite/models/record_merge_criteria.py

@@ -0,0 +1,207 @@
+# Copyright 2019 Digital5 S.L.
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
+
+from odoo import _, fields, models
+from odoo.exceptions import ValidationError
+from odoo.tools.safe_eval import safe_eval
+
+
+class RecordMergeCriteria(models.Model):
+    _name = "record.merge.criteria"
+    _description = "Record Merge by Criteria"
+    _inherit = ["record.merge.mixin"]
+
+    domain = fields.Char(string="Filter", required=True)
+    key = fields.Text(
+        string="Group key",
+        required=True,
+        help="The key that will be used to group the records to merge."
+        "It is based on the object with variable 'o', e.g. o.name.",
+    )
+    group_ids = fields.One2many(
+        comodel_name="record.merge.criteria.group",
+        inverse_name="merge_id",
+        string="Merge Groups",
+    )
+    merge_ids = fields.One2many(
+        comodel_name="record.merge.id", inverse_name="criteria_id", string="Merges"
+    )
+    delete = fields.Selection(
+        string="Default delete",
+        selection=[
+            ("delete", "Delete"),
+            ("deactivate", "Deactivate"),
+            ("recompute", "Recompute"),
+            ("sql", "SQL delete"),
+            ("none", "None"),
+        ],
+        default="delete",
+        required=True,
+    )
+
+    model_name = fields.Char(string="Model name", related="model_id.model")
+
+    merge_relation_id = fields.Many2one(
+        "record.merge.relation.field",
+        string="Origin Relation",
+    )
+    merge_id = fields.Many2one(
+        "record.merge.id",
+        string="Origin Merge",
+        related="merge_relation_id.merge_id",
+        store=True,
+    )
+
+    def get_records_to_merge(self):
+        """
+        get recordset of applied domain
+        :return:
+        """
+        self.ensure_one()
+        model_obj = self.env[self.model_id.model].sudo()
+        domain = safe_eval(self.domain)
+        return model_obj.search(domain)
+
+    def fill_group(self):
+        """
+        fills up the merge groups
+        :return:
+        """
+        for merge in self.filtered(lambda m: m.state == "progress"):
+            model_id = merge.model_id
+            model_obj = self.env[model_id.model]
+            records = merge.get_records_to_merge()
+            group_dict = {}
+            for r in records:
+                key = safe_eval(merge.key, {"o": r})
+                group_dict.setdefault(key, []).append(r.id)
+            merge.group_ids.unlink()
+            lines = []
+            for ids in group_dict.values():
+                if len(ids) > 1:
+                    destiny_record = model_obj.search(
+                        [("id", "in", ids)],
+                        order=merge.order or model_obj._order,
+                        limit=1,
+                    )
+                    lines.append(
+                        (
+                            0,
+                            0,
+                            {
+                                "record_id": destiny_record.id,
+                                "line_ids": [(0, 0, {"record_id": x}) for x in ids],
+                            },
+                        )
+                    )
+            merge.group_ids = lines
+
+    def create_merges(self):
+        """
+        creates merge by ids with the groups
+        :return:
+        """
+        merge_id_obj = self.env["record.merge.id"]
+        for merge in self:
+            if merge.merge_ids.filtered(lambda m: m.state not in ["draft", "cancel"]):
+                raise ValidationError(
+                    _("Cannot delete previous merges, please cancel them.")
+                )
+            merge.merge_ids.unlink()
+            for group in merge.group_ids.filtered(lambda g: g.to_merge):
+                lines = []
+                for line in group.line_ids:
+                    lines.append(
+                        (
+                            0,
+                            0,
+                            {
+                                "record_id": line.record_id,
+                                "destiny": (line.record_id == group.record_id),
+                            },
+                        )
+                    )
+                merge_id_obj.create(
+                    {
+                        "criteria_group_id": group.id,
+                        "model_id": merge.model_id.id,
+                        "name": _("Auto-merge for %s")
+                        % (
+                            ",".join(
+                                [str(x) for x in group.mapped("line_ids.record_id")]
+                            )
+                        ),
+                        "id_line_ids": lines,
+                        # XXX: the order for selecting destiny
+                        # is the order for consolidation?
+                        "order": merge.order,
+                    }
+                )
+
+    # ACTION
+    def action_merge_progress(self):
+        self.write({"state": "progress"})
+        self.fill_group()
+        self.fill_consolidate_field()
+        self.fill_fk_field()
+        self.fill_ref_field()
+        self.fill_nonrel_field()
+        self.fill_rec_field()
+
+    def action_merge_cancel(self):
+        self.write({"state": "cancel"})
+
+    def action_merge_all(self):
+        """
+        launch all processes of all children
+        :return:
+        """
+        self.merge_ids.action_merge_all()
+
+    def action_cancel_all(self):
+        """
+        cancel all children
+        :return:
+        """
+        self.merge_ids.action_merge_cancel()
+
+    def action_merge_done(self):
+        self.write({"state": "done"})
+        self.create_merges()
+
+    def action_refresh_group(self):
+        self.fill_group()
+
+
+class RecordMergeCriteriaGroup(models.Model):
+    _name = "record.merge.criteria.group"
+    _description = "Record Merge Group"
+    _inherit = ["record.merge.mixin.id.line"]
+
+    merge_id = fields.Many2one(
+        "record.merge.criteria", required=True, ondelete="cascade"
+    )
+    record_id = fields.Integer(string="Destiny ID")
+    to_merge = fields.Boolean(default=True)
+    to_delete = fields.Boolean(string="Delete after", default=True)
+    line_ids = fields.One2many(
+        comodel_name="record.merge.criteria.group.line",
+        inverse_name="group_id",
+        string="IDs to Merge",
+    )
+    merge_ids = fields.One2many(
+        comodel_name="record.merge.id",
+        inverse_name="criteria_group_id",
+        string="Merge by ID",
+    )
+
+
+class RecordMergeCriteriaGroupLine(models.Model):
+    _name = "record.merge.criteria.group.line"
+    _description = "Record Merge Group Line"
+    _rec_name = "record_id"
+
+    group_id = fields.Many2one(
+        "record.merge.criteria.group", string="Group", required=True, ondelete="cascade"
+    )
+    record_id = fields.Integer(string="Merge ID", required=True)