pip install torchtext is broken after torch 2.4.0

[sagelywizard]

🐛 Bug

torchtext is in maintenance mode, but there's a problem with the current dependencies which I think may warrant an update and minor version bump. This problem causes pip install torchtext to install a broken installation by default.

Summary

The problem is that the installing the most recent version of torchtext pulls in the latest version of torch (2.4.0) but it's incompatible with that version of torch. Looks like there's some shared object file which is referencing some symbol that was removed. Using torchtext 0.18.0 with torch 2.4.0 causes import torchtext to fail with the following error:

OSError: /usr/local/lib/python3.10/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs

You can repro in a Colab notebook by:

1. uninstalling torch/torchtext. !pip uninstall -y torch torchtext
2. installing torchtext !pip install torchtext.
3. importing torchtext import torchtext

Solution

AFAICT, there are two potential solutions:

1. Update torchtext to be compatible with torch 2.4.0. I suspect this isn't on the table, since torchtext is in maintenance mode.
2. Update torchtext to specify that it requires torch<2.4.0.

Either solution would require a bump in the minor version and a push to pypi.

[atalman]

cc @kartikayk

[stevendbrown]

I am also running into this error with PyTorch 2.5 and 2.5.1, both with torchtext 0.18.0 on two different machines. Maybe I'm showing my ignorance, but I think there might be some issue with how it's getting environment information -- if I try to install from source, I get an error that import torch fails, but it's very much installed in the fresh venv I'm using.

[ZhengPeng7]

I am also running into this error with PyTorch 2.5 and 2.5.1, both with torchtext 0.18.0 on two different machines. Maybe I'm showing my ignorance, but I think there might be some issue with how it's getting environment information -- if I try to install from source, I get an error that import torch fails, but it's very much installed in the fresh venv I'm using.

Hi Steven, @stevendbrown, do you mean that installing torchtext==0.18.0 from source in a fresh venv works well on your machine with torch>2.3.1?

[ZhengPeng7]

Never mind, it worked very well. I installed from source for compatibility with torch==2.7.0 and even generated a whl file, which can be used for easier installations.

[catch-twenty-two]

Rebuilt from source to get a compatible .so binary rebuilt, everything seems to be working now ymmv:

pip uninstall torchtext
git clone https://github.com/pytorch/text.git
cd text
git checkout v0.18.0
git submodule update --init --recursive
python setup.py develop

python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'2.6.0+cu124'
>>> import torchtext
>>> torchtext.__version__
'0.18.0a0+9bed85d'