Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add update_all method #41

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Add update_all method #41

wants to merge 10 commits into from

Conversation

aropie
Copy link

@aropie aropie commented Feb 10, 2025

Overview

Currently, the only way for changes to be reflected in the Autocompleter is to remove_all, then store_all and remove_cache. This makes for a slow and inefficient process because most of the objects weren't really updated.

This PR introduces the method update_all, which in turn calls update_provider for each of the Autocompleter's provider. This method aims to significantly reduce the number of operations made on Redis by bulking them together as much as possible, as well as analyze the data in order to only deal with objects which were actually modified, leaving the rest untouched.
In order to do this, one of the main points was to add a new Redis hashmap djac.provider.sc which stores a mapping of obj_id -> score. This is done in order to have a reference to know when a given object has updated its score, since all objects use the same score across all the ranked sets.

I've tried documenting the method and the reasoning as much as possible, but I agree the whole implementation still feels bulky and somewhat convoluted. Any suggestions on making it leaner/more concise are very welcome

How to test

  • New test cases make sense and pass

@coveralls
Copy link
Collaborator

coveralls commented Feb 13, 2025

Pull Request Test Coverage Report for Build 13312062270

Details

  • 151 of 157 (96.18%) changed or added relevant lines in 4 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.8%) to 93.21%

Changes Missing Coverage Covered Lines Changed/Added Lines %
autocompleter/management/commands/autocompleter_init.py 2 4 50.0%
autocompleter/base.py 147 151 97.35%
Totals Coverage Status
Change from base Build 7672290461: 0.8%
Covered Lines: 961
Relevant Lines: 1031

💛 - Coveralls

@aropie aropie changed the title First attempt at update method Add update_all method Feb 13, 2025
@aropie aropie marked this pull request as ready for review February 13, 2025 17:09
@aropie aropie requested review from anthonyp97 and gabeice February 13, 2025 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants