Python 3.13 incompatibilities

[cansavvy]

Thanks for putting together this cool package!

Summary

I attempted to install gensim 4.3.3 on my mac computer (and on Ubuntu docker images) that were using Python 3.13 and due to some changes in the Python 3.13

I can confirm when I downgraded to Python 3.12 things worked again.

Details

It appears to be because gensim's C code is referring to things that have been deprecated in Python 3.13 See here https://docs.python.org/3/deprecations/index.html

e.g.

• note: 'ma_version_tag' has been explicitly marked deprecated here
• error: no member named 'ob_digit' in 'struct _longobject'

A truncated version of the log output when I try to install with Python 3.13

        251 | #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
            |                                                      ^
      gensim/models/word2vec_inner.c:12176:9: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
       12176 |         __PYX_PY_DICT_LOOKUP_IF_MODIFIED(
             |         ^
      gensim/models/word2vec_inner.c:1697:30: note: expanded from macro '__PYX_PY_DICT_LOOKUP_IF_MODIFIED'
       1697 |         __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\
            |                              ^
      gensim/models/word2vec_inner.c:1686:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
       1686 | #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
            |                                                                 ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/dictobject.h:25:5:
      note: 'ma_version_tag' has been explicitly marked deprecated here
         25 |     Py_DEPRECATED(3.12) uint64_t ma_version_tag;
            |     ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/pyport.h:251:54:
      note: expanded from macro 'Py_DEPRECATED'
        251 | #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
            |                                                      ^
      gensim/models/word2vec_inner.c:12812:55: error: no member named 'ob_digit' in 'struct _longobject'
       12812 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:12867:55: error: no member named 'ob_digit' in 'struct _longobject'
       12867 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:12955:70: error: too few arguments to function call, expected 6, have 5
       12953 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
             |                           ~~~~~~~~~~~~~~~~~~~
       12954 |                                               bytes, sizeof(val),
       12955 |                                               is_little, !is_unsigned);
             |                                                                      ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17:
      note: '_PyLong_AsByteArray' declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^                   ~~~~~~~~~~~~~~~~
        112 |     unsigned char* bytes, size_t n,
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        113 |     int little_endian, int is_signed, int with_exceptions);
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      gensim/models/word2vec_inner.c:13046:55: error: no member named 'ob_digit' in 'struct _longobject'
       13046 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13101:55: error: no member named 'ob_digit' in 'struct _longobject'
       13101 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13189:70: error: too few arguments to function call, expected 6, have 5
       13187 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
             |                           ~~~~~~~~~~~~~~~~~~~
       13188 |                                               bytes, sizeof(val),
       13189 |                                               is_little, !is_unsigned);
             |                                                                      ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17:
      note: '_PyLong_AsByteArray' declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^                   ~~~~~~~~~~~~~~~~
        112 |     unsigned char* bytes, size_t n,
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        113 |     int little_endian, int is_signed, int with_exceptions);
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      gensim/models/word2vec_inner.c:13242:55: error: no member named 'ob_digit' in 'struct _longobject'
       13242 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13297:55: error: no member named 'ob_digit' in 'struct _longobject'
       13297 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13385:70: error: too few arguments to function call, expected 6, have 5
       13383 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
             |                           ~~~~~~~~~~~~~~~~~~~
       13384 |                                               bytes, sizeof(val),
       13385 |                                               is_little, !is_unsigned);
             |                                                                      ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17:
      note: '_PyLong_AsByteArray' declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^                   ~~~~~~~~~~~~~~~~
        112 |     unsigned char* bytes, size_t n,
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        113 |     int little_endian, int is_signed, int with_exceptions);
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      gensim/models/word2vec_inner.c:13438:55: error: no member named 'ob_digit' in 'struct _longobject'
       13438 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13493:55: error: no member named 'ob_digit' in 'struct _longobject'
       13493 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13581:70: error: too few arguments to function call, expected 6, have 5
       13579 |                 int ret = _PyLong_AsByteArray((PyLongObject *)v,
             |                           ~~~~~~~~~~~~~~~~~~~
       13580 |                                               bytes, sizeof(val),
       13581 |                                               is_little, !is_unsigned);
             |                                                                      ^
      /opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/include/python3.13/cpython/longobject.h:111:17:
      note: '_PyLong_AsByteArray' declared here
        111 | PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
            |                 ^                   ~~~~~~~~~~~~~~~~
        112 |     unsigned char* bytes, size_t n,
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        113 |     int little_endian, int is_signed, int with_exceptions);
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      gensim/models/word2vec_inner.c:13634:55: error: no member named 'ob_digit' in 'struct _longobject'
       13634 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      gensim/models/word2vec_inner.c:13689:55: error: no member named 'ob_digit' in 'struct _longobject'
       13689 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
             |                                   ~~~~~~~~~~~~~~~~~~  ^
      fatal error: too many errors emitted, stopping now [-ferror-limit=]
      17 warnings and 20 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1

Suggested solutions:

1. For short term, maybe add warnings and requirements that gensim currently cannot work with Python 3.13
2. For long term, probably need to adjust the items listed above in the code.

[tahaafarooq]

Facing the same issue on python 3.13.2 :(
Thanks @cansavvy 👍🏾

[raffaem]

Also, a binary wheel on PyPi for Python 3.13 on Windows would be appreciated!

[ecederstrand]

Just a +1 from me. Python 3.13 support would be great!

[Tenyama]

+1, had so much trouble figuring out why gensim was stuck on WSL Arch

[julianpollmann]

@cansavvy @ecederstrand @Tenyama could you test with the current dev branch?

[jcaplan]

confirming dev branch with python 3.13

❯ python --version
Python 3.13.3
❯ uv pip list | grep gensim
gensim                                    4.4.0a0.dev0
❯ uv pip list | grep graspologic
graspologic                               3.3.0
graspologic-native                        1.2.5

Checked https://github.com/gusye1234/nano-graphrag works (has dependency on gensim through graspologic)
graspologic-org/graspologic#1091

[ecederstrand]

Confirming that basic testing works here as well.