feat: natural language search #955
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Feature #951
Summary



This PR introduces a natural language command interface for OpenAdapt, allowing users to find and start workflows using simple text queries (e.g., "calculate my expenses") instead of manually searching through a list of recordings. The primary motivation is to make the application more intuitive and accessible, especially for users with many recordings.
The implementation includes:
get_enhanced_similarity_search
function was implemented. It improves search relevance by creating a dynamic context for each recording based on its window titles, action events (like typing), and any transcribed audio, in addition to its title.openadapt/scripts/backfill_embeddings.py
) was created to process existing recordings.Checklist
How can your code be run and tested?
calculate 2+9+11
.Other information
This implementation uses the
sentence-transformers/all-MiniLM-L6-v2
model for generating embeddings, which is now a dependency. The search logic is designed to be self-contained within Python, avoiding the need for external vector database services.