Skip to content

[BUG] Pagination not displaying more than 10k elements and breaks with track_total_hits #311

@pfabri-ebrand

Description

@pfabri-ebrand

Describe the bug
When using the scout pagination, we want to paginate on all the records for a specific model that has more than 10k entries.
The pagination works but stops at 10k, if we add track_total_hits=true to the body it breaks because of the default setting of elastic.

After investigating why, I discovered that to fetch the total amount of records, the system does a request to fetch all the records ids using the total hits as size for the request instead of using the value available in the initial request. This is the extra query that throws an error.

An other issue that will come from this, since the size + from used to paginate cannot be >10k (or the server limit if overwritten) the pagination will break once it goes over that number.

To Reproduce
Steps to reproduce the behavior:

  1. Have a index with more than 10k datas
  2. Create a pagination for the model and add track_total_hits=true to the body sent in the callback
  3. Error will be displayed

Expected behavior
Pagination working for any amount of data, a sort could be provided to use the search_after functionality and do deep pagination

Version
PHP 8.4
Laravel 12.19.3
Scout elastic 7.11.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions