Skip to content

[PERF] import xlsx: avoid reading all cells multiple times during table_conversion #6057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: 16.0
Choose a base branch
from

Conversation

laa-odoo
Copy link
Collaborator

@laa-odoo laa-odoo commented Apr 3, 2025

Previously, the content of each cell was inspected multiple times, once per table, leading to significant performance overhead. In this commit, the inspection process is optimized to traverse the cells only once.

A data structure now stores only the cells that need to be updated, improving both efficiency and calculation time.

This optimization results in a 10% reduction in total import time.

Task: 4699860

@robodoo
Copy link
Collaborator

robodoo commented Apr 3, 2025

Pull request status dashboard

@rrahir
Copy link
Collaborator

rrahir commented Apr 15, 2025

the commit title could point towards the global context in which it is used, which is the Xlsx import if i'm not. mistaken?

…le_conversion

Previously, the content of each cell was inspected multiple times, once per table,
leading to significant performance overhead. In this commit, the inspection process
is optimized to traverse the cells only once.

A data structure now stores only the cells that need to be updated, improving both
efficiency and calculation time.

This optimization results in a 10% reduction in total import time.

task: 4699860
@laa-odoo laa-odoo force-pushed the 16.0-table_conversion-convertTableFormulaReferences-perf-laa branch from 0892c7c to 1d250b1 Compare April 17, 2025 07:51
@laa-odoo
Copy link
Collaborator Author

the commit title could point towards the global context in which it is used, which is the Xlsx import if i'm not. mistaken?

in fact this concerns the xlsx import, the name of the commit has been changed

@laa-odoo laa-odoo changed the title [PERF] table_conversion: avoid reading all cells multiple times [PERF] import xlsx: avoid reading all cells multiple times during table_conversion Apr 17, 2025
Copy link
Collaborator

@LucasLefevre LucasLefevre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it actually slower ? (for the common case where there are no table references)
I exported the large formula dataset to re-import it.
With this branch, convertTableFormulaReferences is 64ms, in 16.0 it's less than 1ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants