Low performance when importing the library

[Social-Mean]

The line from sentence_transformers import SentenceTransformer takes about five seconds to import on my computer. Are there any methods to speed up the import of this library?

[tomaarsen]

Hello!

This approximately matches my import time as well. I'm afraid there's not a convenient fix, especially considering a lot of the import time is due to sentence-transformers importing from various other libraries again (notably torch, transformers).

I ran some profiles using cProfile and snakeviz, but I couldn't find anything that would easily save a lot of import time.

• Tom Aarsen

[Social-Mean]

Hello! I used some profiling tools to analyze the import process and found that the main performance bottlenecks are located at the following points:

1. Recursive calls to the create_import_structure_from_path function in the transformers library, which involve a large number of file system operations;

https://github.com/huggingface/transformers/blob/3aa21543ddda64d24314f1a17d2e80ad8747a9af/src/transformers/utils/import_utils.py#L2415-L2611

2. The _load_dll_libraries function in the torch library, which spends a significant amount of time loading DLL files.

https://github.com/pytorch/pytorch/blob/bebd70e13f6cb34e1d1377b469e89e053c8fb366/torch/__init__.py#L162-L282

[tomaarsen]

Thanks for also looking into this with me. Both of these points make a lot of sense to me. I do suspect that transformers' alternative of not using lazy loading would be a lot slower, so there's not much to be done there. torch is also notorious for being a bit slow to import due to having to load large files, especially when torch is compiled with CUDA.

• Tom Aarsen