Skip to content

Mongodb backend can't handle collections with alot of documents #173

Open
@Halk0

Description

@Halk0

I encountered a bug where querying all documents from collections with lots of documents.
I identified that the problem was how mongodb_filte.py handled filtering. Currently filtering
is done using multiple aggregate queries to mongodb using document ID's that matched
previous filters, this can lead to a situation where a database contains lots of documents
the hard limit of 16 megabyte queries is exceeded and mongodb cannot process the filter. My
changes make it so that instead of multiple filters and queries a single filter is generated
based on the request made and that is passed to mongodb. This way the 16 megabyte query
size limit should never be triggered, and mongodb can process the filter internally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions