Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REF] util.views: refactor converter code, regroup views helpers #23

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

andreabak
Copy link
Contributor

@andreabak andreabak commented Dec 12, 2023

Overview of changes (see commits for details):

  • refactored BootstrapConverter code, separating conversion code and operations from bootstrap-specific code. This way other kind of converters can be implemented, reusing pre-defined ElementOperations into lists (eg. ReplaceClasses/RenameAttributes/etc.)
    • this will also be useful in the (near) future to consolidate into util similar helpers in PS custom-util, commonly used to quickly fix studio and COW views (without all the manual lxml operations boilerplate)
  • generalized helpers and moved from odoo/upgrade base/16.0/pre-90-convert-bootstrap5.py so that they can be reused for other similar kind of operations using converters (ie. apply converter to all html fields / templates, etc.) (upgrade PR: odoo/upgrade#5431 )
  • re-organized views-related helpers into a subpackage util.views (converter, bootstrap, records)
  • misc fixes and tweaks

@robodoo
Copy link
Contributor

robodoo commented Dec 12, 2023

@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch 2 times, most recently from 715851a to 80dc045 Compare December 12, 2023 11:43
@andreabak
Copy link
Contributor Author

upgradeci retry with always all modules in all versions

@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch from 80dc045 to 905ff99 Compare December 14, 2023 13:41
@andreabak andreabak marked this pull request as ready for review February 26, 2024 13:16
@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch from 905ff99 to e7d1a8a Compare February 26, 2024 13:16
@KangOl
Copy link
Contributor

KangOl commented Feb 26, 2024

upgradeci skip

@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch from e7d1a8a to 7b004bc Compare March 5, 2024 11:19
@KangOl
Copy link
Contributor

KangOl commented Mar 5, 2024

upgradeci retry with always only base mass_mailing point_of_sale test_themes theme_common theme_treehouse web web_editor website website_blog website_enterprise website_event website_mass_mailing website_sale in all versions

src/util/__init__.py Outdated Show resolved Hide resolved
src/util/views/records.py Show resolved Hide resolved
src/util/views/__init__.py Outdated Show resolved Hide resolved
src/util/records.py Outdated Show resolved Hide resolved
src/util/modules.py Outdated Show resolved Hide resolved
src/util/__init__.py Outdated Show resolved Hide resolved
src/util/domains.py Outdated Show resolved Hide resolved
@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch 2 times, most recently from a300c58 to 83c4db4 Compare March 6, 2024 08:06
src/util/records.py Outdated Show resolved Hide resolved
src/util/records.py Outdated Show resolved Hide resolved
@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch from 83c4db4 to 767a572 Compare March 6, 2024 14:55
@andreabak
Copy link
Contributor Author

upgradeci retry

@andreabak andreabak force-pushed the master-util_refactor_view_helpers-abt branch from 767a572 to 9173eb0 Compare May 15, 2024 08:21
@andreabak
Copy link
Contributor Author

@KangOl I've rebased this on the latest master commit, resolved conflicts, etc.. CI checks seem good.
Is it ready to go? (together with https://github.com/odoo/upgrade/pull/5431)

FYI @aj-fuentes

andreabak and others added 6 commits May 17, 2024 15:32
Refactored BootstrapConverter code, separating conversion class and
operations from bootstrap-specific code.
The new EtreeConverter can be used to implement conversions of any
kind, reusing the pre-defined ElementOperations by defining an
operations list.
Simplified ElementOperation class, removing unnecessary features, such
as `.op()` method.
Made `xpath` attribute part of the base class, subclasses must honor
it to restrict their scope of operation.

Reworked EtreeConverter class methods, generalized to work with any
kind of converter operations (not just Bootstrap conversions).

Also made docstrings private (excluded from online docs).
Move xpath keywords / where clause functions from base/16.0
pre-90-convert-bootstrap5.py script into util.views and add shortcut
methods for them in converter class. Also refactored to make them
usable with arbitrary converters.

Revise converter API, especially simplified classmethods / remove
oneshot aliases (it's easier to just instantiate the converter first).

Refactor parse/unparse string arch code into separate helper
functions, and add a `ArchEditor` context manager to
parse-edit-unparse an arch within a context (similar to edit_views).

Make sure EtreeConverter is pickle-able for multiprocessing:
concurrent.futures ProcessPoolExecutor uses multiprocessing to spawn
its subprocess workers. That requires memory data from the main
process to be transferred by serializing and deserializing, using the
pickle library. To make this work:
- the converter function must not be an anonymous "factory" function,
  e.g. one generated as a nested function, so unaccessible from the
  outer scope.
- instance values in the serialized objects must also be pickleable.
Therefore the following changes have been done:
- the converter function is now a method of the converter class
- to keep @lru_cache decorator on the method, the entire class is
  made hashable, this is done by computing a hash of the provided
  arguments to `__init__`, making some attributes "private", and
  adding read-only properties to access them.
- to make the instances pickable, the compiled coversions are first
  removed from the pickle-able `__dict__`, because `lxml.XPath`
  objects are not python-native, and they're re-compiled when the
  instances are de-serialized.

Made docstrings for new functions private (excluded from online docs).

upgrade PR: odoo/upgrade#5431
Changes:
- move `util.views_convert` -> `util.views.convert`
- move records-related views helpers into `util.views.records`
- move `util.convert_bootstrap` -> `util.views.bootstrap`
- move some helpers from `util.records` -> `util.views.records`
- add version checks for views/html conversion helpers
Store and return the number of matched and converted record's values
that are processed by `convert_html_columns()`.
Add a `verbose=False` optional argument in `convert_html_content()`
that collects these stats and logs them.

Also, do the same in `util.views.records.convert_html_fields()`.
`util.snippets` imports util from `odoo.upgrade` namespace package
and that makes it incompatible with odoo versions <= 13.0.

Replaced the import with direct package-relative ones.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants