[RMP] Support for simple ML/CF Models (like Implicit) in Merlin Models and Systems

[viswa-nvidia]

Problem:

Latent factor models enable discovery of the underlying structure between interactions and items. These approaches have been popular over the years to leverage implicit feedback data. Customers who are using simple models via Implicit and LightFM want to be able to deploy those models within the Merlin ecosystem.

Goal:

• Provide training support for implicit and lightfm based models in Merlin Models
• Provide inference support for implicit based models in Merlin Systems
• Provide inference support for lightfm based models in Merlin Systems systems#200

Constraints:

Systems

• Serve Implicit/LightFM as a self-contained op with everything required to serve within the exported triton model directory
  Requires installing the python package in the tritonserver environment where it will run.
  Decomposing the serving of these into different operators (retrieval with nearest neighbour search through embedding space)

Blocking issues

• Inference is blocked w/ issues on serialization of model w/ Implicit

Starting Point:

Merlin-models

wrap Implicit and lightFM in High-level model API:

• Add shim to adapt the implicit library models#218
• Add shim to adapt the lightfm library models#219
  Note: 218 and 219 only builds the models. It doesn't take care of serving. It doesn't have the example.
• Tweak wrapper for Implicit to account for change surfaced by paper experiments models#410
  Note: Scope currently unclear (@oliverholworthy )

NVTabular

N/A

Merlin-systems

• Add operators for LightFM models systems#35
  • Implicit operator (self-contained)
    Add Operator for Implicit Models systems#134
  • Support for simple ML/CF Models - LightFM serving #457

Examples and Docs (To happen in 22.09)

• [Task] Example of training Implicit in Merlin Models  models#561
• Example of serving Implicit in Merlin Systems models#577
• Example of training LightFM in Merlin Models models#576
• Example of serving LightFM in Merlin Systems models#578

[viswa-nvidia]

@benfred , please add the tickets related to this epic here

[viswa-nvidia]

@EvenOldridge @benfred , is Batch prediction in NVTabular section required for 22.04 ?

[oliverholworthy]

For the serving part. My understanding is that, for both Implicit and LightFM, the way to serve these at scale is similar to the two tower / bi-encoder design of the DLRM example we have.

Where the item embeddings are stored in a approximate-nearest neighbour search index. And we search through this space using a user embedding. (for user->item recommendations).

The difference from the DLRM example, is in how the user embedding is computed at serving time.

Computing the User Embedding:

• LightFM represents users and items as an aggregation (sum) of their feature embeddings. In this case we would lookup all the feature embeddings corresponding to the user and then aggregate (sum) these.
• Implicit has support for a few different models. Each has the concept of an item and user (factor/embedding)
  The alternating least-squares implementation (ALS) appears to be the only one that supports calculating the user factor (embedding) for unseen/new users by running the solver provided items the user has interacted and some model parameters (I'm currently unsure if it would be feasible to store this in practice in a serving scenario / how fast this inference would be).
  However, for the case where we have the user factors/embeddings computed already, we can look these up at query-time.

It seems that one of the main questions is where do we store the user/item/feature embeddings for fast lookup at serving time. Ideally with support for versioning (for migrating to new model version) / updating (for new users and items).

For vector/embedding based item->user, user->user, item->item recommendations. two main components are:

• Storing user/item/feature embeddings (Key-value lookup) + optional aggregation (e.g. in the case of LightFM)
• Approximate Nearest Neighbour search through item or user space.
  We currently have the QueryFaiss operator. Which is currently limited to a pre-computed index. (so would need to re-compute the index and re-deploy Triton for new items to be considered.) -> Issue for extending functionality of the nearest neighbour candidate retrieval here Add integrations for other nearest neighbor search tools systems#10

We could start with a similar approach to the nearest neighbour implementation we have now. A first pass at this could be to store the user/feature embeddings as static data inside Triton. Albeit with similar challenges to having the item embeddings index stored as file(s) inside Triton (updating, and scaling to large numbers of items/users). Or they could be managed in a database outside Triton.