Skip to content

Add InnerProduct and CosineExpanded metric support in NN Descent #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 37 commits into from
Nov 15, 2024
Merged

Add InnerProduct and CosineExpanded metric support in NN Descent #177

merged 37 commits into from
Nov 15, 2024

Conversation

divyegala
Copy link
Member

@divyegala divyegala commented Jun 5, 2024 •
edited
Loading

Closes #171

@divyegala divyegala added feature request New feature or request non-breaking Introduces a non-breaking change labels Jun 5, 2024
@divyegala divyegala self-assigned this Jun 5, 2024
@divyegala divyegala requested review from a team as code owners June 5, 2024 22:20
@divyegala divyegala changed the title Add InnerProduct metric support in NN Descent Add InnerProduct and CosineExpanded metric support in NN Descent Jul 13, 2024
@divyegala divyegala changed the base branch from branch-24.08 to branch-24.10 September 9, 2024 19:18
@divyegala divyegala changed the title Add InnerProduct and CosineExpanded metric support in NN Descent Add InnerProduct metric support in NN Descent Sep 9, 2024
@divyegala divyegala requested a review from a team as a code owner October 31, 2024 23:17
@divyegala divyegala requested a review from AyodeAwe October 31, 2024 23:17
@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@divyegala divyegala changed the base branch from branch-24.10 to branch-24.12 October 31, 2024 23:17
@divyegala divyegala removed request for a team and AyodeAwe October 31, 2024 23:18
@github-actions github-actions bot removed the CMake label Oct 31, 2024
@divyegala divyegala removed the request for review from a team October 31, 2024 23:19
@github-actions github-actions bot added the CMake label Nov 1, 2024
@divyegala divyegala changed the title Add InnerProduct metric support in NN Descent Add InnerProduct and CosineExpanded metric support in NN Descent Nov 12, 2024
cjnolet
cjnolet reviewed Nov 12, 2024
View reviewed changes
cjnolet
cjnolet reviewed Nov 12, 2024
View reviewed changes
@divyegala
Copy link
Member Author

divyegala commented Nov 12, 2024 •
edited
Loading

@cjnolet here's some quick benchmark numbers. I printed the metric out to ensure the right one is being passed through from the benchmarks. IMO the impact of the division and sqrt is minimal but perhaps you feel differently:

metric: cosine
[I] [15:30:05.587420] optimizing graph
[I] [15:30:06.510756] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       12.0 s           158 s             1    11.9721           32      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:30:06.884944] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: cosine
[I] [15:30:17.943359] optimizing graph
[I] [15:30:19.465069] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       12.7 s           161 s             1    12.6604           32      1000k                        96 graph_build_algo="NN_DESCENT"
[I] [15:30:19.844759] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT
metric: cosine
[I] [15:30:30.708098] optimizing graph
[I] [15:30:32.099818] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       12.4 s           178 s             1    12.3581           64      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:30:32.782952] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: cosine
[I] [15:30:44.454569] optimizing graph
[I] [15:30:46.214032] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       13.5 s           194 s             1    13.5414           64      1000k                        96 graph_build_algo="NN_DESCENT"

metric: euclidean
[I] [15:33:53.323826] optimizing graph
[I] [15:33:54.058909] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       11.8 s           148 s             1    11.7708           32      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:33:54.419808] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: euclidean
[I] [15:34:05.632598] optimizing graph
[I] [15:34:06.974172] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       12.6 s           160 s             1    12.6361           32      1000k                        96 graph_build_algo="NN_DESCENT"
[I] [15:34:07.347230] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT
metric: euclidean
[I] [15:34:18.181052] optimizing graph
[I] [15:34:19.455167] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       12.2 s           166 s             1    12.2109           64      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:34:20.156899] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: euclidean
[I] [15:34:31.725188] optimizing graph
[I] [15:34:33.502260] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       13.5 s           192 s             1    13.4508           64      1000k                        96 graph_build_algo="NN_DESCENT"

metric: inner_product
[I] [15:43:30.422822] optimizing graph
[I] [15:43:31.238908] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       11.9 s           155 s             1    11.9286           32      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:43:32.618730] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:43:43.869825] optimizing graph
[I] [15:43:45.203679] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       12.7 s           174 s             1    12.6838           32      1000k                        96 graph_build_algo="NN_DESCENT"
[I] [15:43:45.598379] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:43:56.582574] optimizing graph
[I] [15:43:57.889899] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       12.4 s           177 s             1    12.4257           64      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:43:58.604195] Overwriting file: /raid/dgala/cuvs/datasets/sift-128-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:44:09.842805] optimizing graph
[I] [15:44:11.647437] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       13.2 s           177 s             1    13.1674           64      1000k                        96 graph_build_algo="NN_DESCENT"

@divyegala
Copy link
Member Author

gist-960-euclidean numbers:

metric: euclidean
[I] [15:50:56.395567] optimizing graph
[I] [15:50:57.138982] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       32.5 s           174 s             1    32.4557           32      1000k                        64 graph_build_algo="NN_DESCENT"
metric: euclidean
[I] [15:51:28.327044] optimizing graph
[I] [15:51:29.536358] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       32.2 s           190 s             1    32.2073           32      1000k                        96 graph_build_algo="NN_DESCENT"
metric: euclidean
[I] [15:52:01.424672] optimizing graph
[I] [15:52:02.593928] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       33.0 s           188 s             1    32.9537           64      1000k                        64 graph_build_algo="NN_DESCENT"
metric: euclidean
[I] [15:52:34.354768] optimizing graph
[I] [15:52:35.988628] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       33.0 s           198 s             1    33.0033           64      1000k                        96 graph_build_algo="NN_DESCENT"

metric: inner_product
[I] [15:56:24.627861] optimizing graph
[I] [15:56:25.362416] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       32.4 s           167 s             1    32.4496           32      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:56:26.244176] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:56:56.738775] optimizing graph
[I] [15:56:57.905511] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       32.3 s           193 s             1    32.2539           32      1000k                        96 graph_build_algo="NN_DESCENT"
[I] [15:56:58.783456] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:57:29.642376] optimizing graph
[I] [15:57:30.942815] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       32.6 s           198 s             1    32.6481           64      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [15:57:32.024089] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: inner_product
[I] [15:58:02.179546] optimizing graph
[I] [15:58:03.901965] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       32.5 s           210 s             1    32.4916           64      1000k   

metric: cosine
[I] [16:01:29.537623] optimizing graph
[I] [16:01:30.389369] Graph optimized, creating index
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                        Time             CPU   Iterations        GPU graph_degree index_size intermediate_graph_degree
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
cuvs_cagra.graph_degree32.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       32.5 s           183 s             1    32.4771           32      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [16:01:31.237660] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: cosine
[I] [16:02:01.286174] optimizing graph
[I] [16:02:02.452481] Graph optimized, creating index
cuvs_cagra.graph_degree32.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       31.7 s           180 s             1    31.7016           32      1000k                        96 graph_build_algo="NN_DESCENT"
[I] [16:02:03.236910] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT
metric: cosine
[I] [16:02:34.056109] optimizing graph
[I] [16:02:35.229266] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree64.graph_build_algoNN_DESCENT/process_time/real_time       32.5 s           189 s             1    32.4824           64      1000k                        64 graph_build_algo="NN_DESCENT"
[I] [16:02:36.295483] Overwriting file: /raid/dgala/cuvs/datasets/gist-960-euclidean/index/cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT
metric: cosine
[I] [16:03:06.931169] optimizing graph
[I] [16:03:08.605045] Graph optimized, creating index
cuvs_cagra.graph_degree64.intermediate_graph_degree96.graph_build_algoNN_DESCENT/process_time/real_time       32.8 s           215 s             1    32.8123           64      1000k                        96 graph_build_algo="NN_DESCENT"

@divyegala
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 7ab2bfd into rapidsai:branch-24.12 Nov 15, 2024
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cjnolet cjnolet cjnolet approved these changes

Assignees

@divyegala divyegala

Labels
CMake cpp feature request New feature or request non-breaking Introduces a non-breaking change Python
Projects
Vector Search, ML, & Data Mining Release Board
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Enable inner product distances for NN Descent
2 participants
@divyegala @cjnolet