Extras-require, base package without Faiss-CPU requirement

[raphaelsty]

Hi,

I appreciate the work you're doing with the Beir benchmark – it's very useful.

I'm developing a library that enables training models like Splade, Sparsembed, BLP, and Colbert using PyTorch. I've included Beir in the requirements to facilitate evaluations during training. However, I'm facing an issue. I want to utilize Faiss GPU with my own retriever for evaluating my models on Beir benchmarks and I don't wan't to install Faiss CPU.

I'm wondering if it's possible to create a custom extras_requires section in your setup.py that would exclude Faiss for users who don't intend to use Beir retrievers.

Here is an example:

pip install "beir[no_faiss]"

base_packages = [
      'sentence-transformers',
      'pytrec_eval',
      'faiss_cpu',
      'elasticsearch==7.9.1',
      'datasets'
 ]

no_faiss = [
      'sentence-transformers',
      'pytrec_eval',
      'elasticsearch==7.9.1',
      'datasets'
 ]

setuptools.setup(
    ...,
    packages=setuptools.find_packages(),
    install_requires=base_packages,
    extras_require={
        "no_faiss": no_faiss
    }
)

I'm fine with creating a PR if you are ok with this.

Thank you,

Raphaël

[thakur-nandan]

Hi @raphaelsty, Sounds like a meaningful suggestion.

Feel free to create the PR. I can go ahead and merge the PR.

Regards,
Nandan