-
-
Notifications
You must be signed in to change notification settings - Fork 123
Description
Describe the bug
When using the scout pagination, we want to paginate on all the records for a specific model that has more than 10k entries.
The pagination works but stops at 10k, if we add track_total_hits=true
to the body it breaks because of the default setting of elastic.
After investigating why, I discovered that to fetch the total amount of records, the system does a request to fetch all the records ids using the total hits as size
for the request instead of using the value available in the initial request. This is the extra query that throws an error.
An other issue that will come from this, since the size
+ from
used to paginate cannot be >10k (or the server limit if overwritten) the pagination will break once it goes over that number.
To Reproduce
Steps to reproduce the behavior:
- Have a index with more than 10k datas
- Create a pagination for the model and add
track_total_hits=true
to the body sent in the callback - Error will be displayed
Expected behavior
Pagination working for any amount of data, a sort could be provided to use the search_after
functionality and do deep pagination
Version
PHP 8.4
Laravel 12.19.3
Scout elastic 7.11.2