gensim is not longer usable due to pinning scipy <1.14.0

[morotti]

hello,

reporting that gensim is no longer installable due to pinning an old scipy<1.14.0

• gensim is no longer installable on recent python versions that only have a recent version of scipy.
• gensim is no longer installable with other recent packages that require scipy 1.14, which is quite a few.

I note that scipy 1.11 1.12 1.13 are crashing on skylake / cascade lake Intel CPUs, they call invalid instructions and crash depending on what matrix operations you do, so the upgrade to scipy 1.14 is quite critical.

the pin is in the setup.py with a comment https://github.com/piskvorky/gensim/blob/master/setup.py#L337

install_requires = [
    ...
    # scipy 1.14.0 and onwards removes deprecated sparsetools submodule
    'scipy >= 1.7.0, <1.14.0',

[morotti]

fix, patching the 4.3.3 release wheel, tested on scipy 1.14.

it seems it's just one import that moved under scipy.sparse.sparsetools.

diff -uprNB gensim-4.3.3-original/gensim/models/lsimodel.py gensim-4.3.3-fix/gensim/models/lsimodel.py
--- gensim-4.3.3-original/gensim/models/lsimodel.py	2025-07-28 12:22:16.548067466 +0100
+++ gensim-4.3.3-fix/gensim/models/lsimodel.py	2025-07-28 12:31:53.088791674 +0100
@@ -66,7 +66,12 @@ import time
 import numpy as np
 import scipy.linalg
 import scipy.sparse
-from scipy.sparse import sparsetools
+
+try:
+    from scipy.sparse import csc_matvecs
+except ImportError:
+    # the function is no longer public in scipy 1.14+
+    from scipy.sparse._sparsetools import csc_matvecs
 
 from gensim import interfaces, matutils, utils
 from gensim.models import basemodel
@@ -960,7 +965,7 @@ def stochastic_svd(
         m, n = corpus.shape
         assert num_terms == m, f"mismatch in number of features: {m} in sparse matrix vs. {num_terms} parameter"
         o = random_state.normal(0.0, 1.0, (n, samples)).astype(y.dtype)  # draw a random gaussian matrix
-        sparsetools.csc_matvecs(
+        csc_matvecs(
             m, n, samples, corpus.indptr, corpus.indices,
             corpus.data, o.ravel(), y.ravel(),
         )  # y = corpus * o
@@ -994,7 +999,7 @@ def stochastic_svd(
             num_docs += n
             logger.debug("multiplying chunk * gauss")
             o = random_state.normal(0.0, 1.0, (n, samples), ).astype(dtype)  # draw a random gaussian matrix
-            sparsetools.csc_matvecs(
+            csc_matvecs(
                 m, n, samples, chunk.indptr, chunk.indices,  # y = y + chunk * o
                 chunk.data, o.ravel(), y.ravel(),
             )
diff -uprNB gensim-4.3.3-original/gensim-4.3.3.dist-info/METADATA gensim-4.3.3-fix/gensim-4.3.3.dist-info/METADATA
--- gensim-4.3.3-original/gensim-4.3.3.dist-info/METADATA	2025-07-28 12:22:18.317676494 +0100
+++ gensim-4.3.3-fix/gensim-4.3.3.dist-info/METADATA	2025-07-28 12:23:30.427151029 +0100
@@ -26,7 +26,7 @@ Classifier: Topic :: Text Processing ::
 Requires-Python: >=3.8
 License-File: COPYING
 Requires-Dist: numpy <2.0,>=1.18.5
-Requires-Dist: scipy <1.14.0,>=1.7.0
+Requires-Dist: scipy
 Requires-Dist: smart-open >=1.8.1
 Provides-Extra: distributed
 Requires-Dist: Pyro4 >=4.27 ; extra == 'distributed'

[piskvorky]

Thanks. @julianpollmann if that works it should be much better (more efficient) than the dense replacement in #3615.

[julianpollmann]

@piskvorky @morotti Yes, I added the suggestion. Thanks!

[Clayblockunova]

@julianpollmann have you ever considered for opening a PR for that?

[julianpollmann]

@Clayblockunova this should be fixed with the merge of #3615. Can you test with the dev branch? There will be a new release with this fix soon.