How are faiss metrics collected?

[Lok-Yat-man]

Question

I'm going to train and query vectors with faiss IVFFlat, and then collect metrics: quantization time, search time, number of distance calculations, average latency. When I was reading the source code, I found a struct IndexIVFStats that had the metrics I needed, but I couldn't find a function to get those properties. Is there any way to count these metrics?

My code

    faiss.omp_set_num_threads(16)
    d = base_vectors.shape[1]
    k_clusters = int(len(base_vectors) * 0.01)
    nprobe = 1
    top_k = 20
    
    quantizer = faiss.IndexFlatL2(d)
    index = faiss.IndexIVFFlat(quantizer, d, k_clusters, faiss.METRIC_L2)

    train_start = time.time()
    index.train(base_vectors)
    train_time = time.time() - train_start

    add_start = time.time()
    index.add(base_vectors)
    add_time = time.time() - add_start

    index.nprobe = nprobe 
    search_start = time.time()
    _, labels = index.search(query_vectors, top_k)
    search_time = time.time() - search_start
    
    total_time = train_time + add_time + search_time

Platform

OS: Ubuntu22

faiss version: 1.9

Running on：

• CPU
• GPU

Language：

• Python
• C++

[Lok-Yat-man]

I found a function that did what I needed: stats = faiss.cvar.indexIVF_stats, stats is a global variable that can get the IndexIVFStats struct:

struct IndexIVFStats {
    size_t nq;                // nb of queries run
    size_t nlist;             // nb of inverted lists scanned
    size_t ndis;              // nb of distances computed
    size_t nheap_updates;     // nb of times the heap was updated
    double quantization_time; // time spent quantizing vectors (in ms)
    double search_time;       // time spent searching lists (in ms)

    IndexIVFStats() {
        reset();
    }
    void reset();
    void add(const IndexIVFStats& other);
};

If you want to use stats more than once, refresh it with faiss.cvar.indexIVF_stats.reset()