Skip to content

[IMP] filter: fix horrible performances with huge data filters #6286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hokolomopo
Copy link
Contributor

On a spreadsheet with 10,000 rows, opening the filter menu and filtering all the values is really slow.

This commit improves the performance of the existing functions, and adds a load more button in the filter menu to avoid creating 10000 owl components and DOM elements.

On a data filter with 10,000 rows:

  • getFilterHiddenValues (when opening filter menu): ~1000ms => ~70ms
  • updateHiddenRows (when filtering all values): ~543ms => ~8ms

Task: 4658998

review checklist

  • feature is organized in plugin, or UI components
  • support of duplicate sheet (deep copy)
  • in model/core: ranges are Range object, and can be adapted (adaptRanges)
  • in model/UI: ranges are strings (to show the user)
  • undo-able commands (uses this.history.update)
  • multiuser-able commands (has inverse commands and transformations where needed)
  • new/updated/removed commands are documented
  • exportable in excel
  • translations (_t("qmsdf %s", abc))
  • unit tested
  • clean commented code
  • track breaking changes
  • doc is rebuild (npm run doc)
  • status is correct in Odoo

@robodoo
Copy link
Collaborator

robodoo commented May 5, 2025

This PR targets the un-managed branch odoo/o-spreadsheet:master-data-filter-rule-adrm, it needs to be retargeted before it can be merged.

@hokolomopo hokolomopo force-pushed the master-data-filter-rule-adrm branch 2 times, most recently from d89b946 to 55e57a3 Compare May 7, 2025 14:15
@rrahir rrahir force-pushed the master-data-filter-rule-adrm branch from 55e57a3 to 8fdbfad Compare May 12, 2025 12:02
On a spreadsheet with 10,000 rows, opening the filter menu and
filtering all the values is really slow.

This commit improves the performance of the existing functions, and
adds a `load more` button in the filter menu to avoid creating
10000 owl components and DOM elements.

On a data filter with 10,000 rows:
- `getFilterHiddenValues` (when opening filter menu): ~1000ms => ~70ms
- `updateHiddenRows` (when filtering all values): ~543ms => ~8ms

Task: 4658998
@hokolomopo hokolomopo force-pushed the master-data-filter-huge-dataset-adrm branch from 5ea8849 to 4e47bc9 Compare May 12, 2025 13:25
@hokolomopo hokolomopo changed the base branch from master-data-filter-rule-adrm to master May 12, 2025 13:26
@robodoo
Copy link
Collaborator

robodoo commented May 12, 2025

Pull request status dashboard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants