Error installing: ValueError: numpy.dtype size changed

[BryceStevenWilley]

Having a lot of trouble installing FormFyxer on a fresh virtual env. Below is the error

    File "/tmp/pip-build-env-jyabgwzo/overlay/lib/python3.10/site-packages/thinc/backends/__init__.py", line 17, in <module>
     from .cupy_ops import CupyOps
  File "/tmp/pip-build-env-jyabgwzo/overlay/lib/python3.10/site-packages/thinc/backends/cupy_ops.py", line 16, in <module>
     from .numpy_ops import NumpyOps
   File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Related to explosion/spaCy#13528, but I seem to be having the same error when I fix the version of numpy to below 2.0.0. Not sure why the obvious fix isn't working, but pip runs all of it in a temporary dir that gets deleted, so I can't see what actual code is being run there.

Workaround is just to remove the entire InstallSpacyModelCommand:

FormFyxer/setup.py

Lines 9 to 14 in fce8447

	class InstallSpacyModelCommand(install):
	def run(self):
	install.run(self)
	import spacy
	print("Downloading word2vec model en_core_web_sm")
	spacy.cli.download('en_core_web_sm')

That workaround won't fly for our situation, but I'll have to see if it breaks our GH actions as well, or if it's just on my machine.

[BryceStevenWilley]

An example of the build failure: https://github.com/SuffolkLITLab/FormFyxer/actions/runs/9540258846/job/26291740248?pr=136

[BryceStevenWilley]

Figured out a local fix: from the thinc docs, you can specify build constraints. I think since we're running spacy while building/installing formfyxer, the normal fix of "just use numpy<2.0.0" doesn't work because of how pip installs dependencies when building packages. Can't fully articulate, but if you run:

PIP_CONSTRAINT=build-constraints.txt pip install . --no-cache-dir

with build-constraints.txt as:

numpy==1.26.4

it works.

Happy to hear additional approaches, but I think the best way to fix this at the moment is to patch ALActions to have a similar build-constraints.txt in it, and then use it when building everything. It shouldn't matter for most repos, but I think this spacy / thinc / numpy issue will pop up in a few different places.

[BryceStevenWilley]

Okay, with SuffolkLITLab/ALActions#26, this should be fixed on CI/CD. Not sure if we should add something to the docs saying that pip install . won't work though.

TBH, I'm sure this will start affecting a few people come this week, and spacy and thinc might come up with better workarounds, so I might keep this open just to wait for those.