Skip to content

[STORY] Search improvments ? #1760

@chibenwa

Description

@chibenwa
Member

Point 1: In Twake Mail I want to have a different OpenSearch mapping / query pattern than in James.

(Ease of change, project independance, etc...)

  • Have an interface for the mapping and for the CriterionConverter to ease overrides.

Point 2 Enable "prefix search" but limit it to the subject

-> Why: can improve search result efficiency (?)

But we limit it to subject (a small field) as ngram use is prone to x10-50 space amplicication
(we may afford this only on the subject)

The idea would be to have :

  • 1 indexed subfield with the standard analyzer like today
  • 1 indexed subfield with the ngrams

If the search contains a space then we use only standard analyzer (heuristic here: if theres a space its not a prefix) but if not we look for both.

We need to have a configuration to enable point 2, disabled by default.

To be adopted on LNG first. Create the new index (empty) and do a per user reindexing on all users.

Adopt on TWP preprod + prod too.

Sub-issues

Sub-issues

3 of 3 Issues completed

Activity

chibenwa

chibenwa commented on May 19, 2025

@chibenwa
MemberAuthor

The idea:

Image

And of course adapt the CriterionConverter logic to this.

Arsnael

Arsnael commented on May 30, 2025

@Arsnael
Member

Quick brainstorm on Point 1:

Mailbox mapping:
In James =>

  • Interface => MailboxMappingFactory with method declaration getMappingContent()
  • MailboxMapping => implements MailboxMappingFactory
  • guice bindings

In TMail => have a TMailMailboxMapping implements MailboxMappingFactory with a different mapping

CriterionConverter:
In James =>

  • Interface => CriterionConverterFactory with method declaration convertCriterion(Criterion criterion)
  • CriterionConverter => implements CriterionConverterFactory
  • guice bindings

In TMail => have a TMailCriterionConverter with a different logic

quantranhong1999

quantranhong1999 commented on May 30, 2025

@quantranhong1999
Member

Quick brainstorm on Point 1:

Sounds reasonable to me. I had a look at MessageToOpenSearchJson BTW. We do not need to interface it because likely for now, we still index the same JSON document, just with a different analyzer.

quantranhong1999

quantranhong1999 commented on May 30, 2025

@quantranhong1999
Member

For point 2, it would use what was set up thanks to point 1.

In TMailMailboxMapping, for the subject field:

  • 1 indexed subfield with the standard analyzer like today
  • 1 indexed subfield with the ngrams

In TMailCriterionConverter, implement this extra logic for subject search query: If the search contains a space then we use only standard analyzer (heuristic here: if theres a space its not a prefix) but if not we look for both.

We need to have a configuration to enable point 2, disabled by default.

We can do a configuration and module chooser IMO. If point 2 enabled, we load TMailMailboxMapping + TMailCriterionConverter, otherwise we load what it is today in James.

To be adopted on LNG first. Create the new index (empty) and do a per user reindexing on all users.

We need to add an entry in upgrade-instructions.md to document the process.

chibenwa

chibenwa commented on May 30, 2025

@chibenwa
MemberAuthor

Sounds reasonable to me. I had a look at MessageToOpenSearchJson BTW. We do not need to interface it because likely for now, we still index the same JSON document, just with a different analyzer.

IMO we can keep the same JSON document

In TMailMailboxMapping, for the subject field:

1 indexed subfield with the standard analyzer like today
1 indexed subfield with the ngrams

+1 this is what I had in mind

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @chibenwa@Arsnael@quantranhong1999

        Issue actions

          [STORY] Search improvments ? · Issue #1760 · linagora/tmail-backend