Skip to content

feat: Add OnlineStore for MongoDB#6025

Open
caseyclements wants to merge 29 commits intofeast-dev:masterfrom
caseyclements:INTPYTHON-297-MongoDB-Feast-Integration
Open

feat: Add OnlineStore for MongoDB#6025
caseyclements wants to merge 29 commits intofeast-dev:masterfrom
caseyclements:INTPYTHON-297-MongoDB-Feast-Integration

Conversation

@caseyclements
Copy link

@caseyclements caseyclements commented Feb 26, 2026

What this PR does / why we need it:

Adds a first-class MongoDB online store integration at feast.infra.online_stores.mongodb_online_store.

Schema

Each entity is stored as a single MongoDB document keyed by its serialized entity key:

{
  "_id": "<serialized_entity_key>",
  "features": { "<feature_view>": { "<feature>": <value> } },
  "event_timestamps": { "<feature_view>": "<datetime>" },
  "created_timestamp": "<datetime>"
}

Because MongoDB has a loose schema and supports upsert semantics natively, update() requires no
pre-creation — it only removes feature views named in tables_to_delete via $unset. Multiple
feature views for the same entity share one document.

Implementation highlights

  • Sync and async paths — implements online_write_batch, online_read,
    online_write_batch_async, online_read_async, and async def close().
    async_supported returns read=True, write=True.
  • requested_features projection — both read paths build a MongoDB field projection so only
    the requested feature columns are returned from the server.
  • Efficient BSON → proto conversion_convert_raw_docs_to_proto transforms column-wise to
    minimise calls to python_values_to_proto_values (one call per feature across all entities,
    rather than one call per entity × feature). Benchmarking confirmed this is ~3–4× faster than a
    naïve row-wise approach across all scaling dimensions (entities, features, feature views).
  • Composite entity keys — entities are serialized with serialize_entity_key, so composite
    keys (e.g. customer_id + driver_id) are handled without schema changes.

Tests

  • Docker integration test (test_mongodb_online_features) — spins up a real MongoDB container
    via testcontainers, writes to three feature views (single int key, single string key, composite
    key), and asserts correct retrieval including type coercion and missing-entity handling.
  • Pure Python unit tests (no Docker required):
    • test_convert_raw_docs_missing_entity — entity absent from query results → (None, None)
    • test_convert_raw_docs_partial_doc — entity present but a feature key missing → empty
      ValueProto (schema-migration safety)
    • test_convert_raw_docs_ordering — result order follows the requested ids list regardless
      of MongoDB cursor order

All tests pass. Code is clean under mypy, ruff check, and ruff format.


Open with Devin

… pymongo is found as extra

Signed-off-by: Casey Clements <[email protected]>
…lineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.

Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
…r now. simply and transformming

Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
… to proto, removed the naive one. It was outcompeted 3X across dimensions

Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
Signed-off-by: Casey Clements <[email protected]>
@caseyclements caseyclements requested a review from a team as a code owner February 26, 2026 00:30
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Casey Clements <[email protected]>
@caseyclements caseyclements force-pushed the INTPYTHON-297-MongoDB-Feast-Integration branch from 5a815be to 6581159 Compare February 27, 2026 21:46
@shuchu
Copy link
Collaborator

shuchu commented Feb 28, 2026

@caseyclements please check the errors in the tests :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants