ValueError: Unpack failed: incomplete input - when custom ner is loaded on Linux server. All works perfectly fine on my local host - windows 11

[rumnen]

Similar issue was discussed before here: #10016

Despite that I receive same error ValueError: Unpack failed: incomplete input, I believe that this case is quite different as the error occurs only on Linux server and when custom ner is loaded.

I tried to install on the Linux server different versions and their related models. Same code, with the same method of installation, same versions of the package and model work perfectly fine on my PC.

pip install spacy
python -m spacy download en_core_web_sm

here are some OS details:
Linux kernel version 4.18.0-425.13.1.lve.el8.x86_64,
Apache version 2.4.57

I can post the entire log of the traceback plus my code if that would be helpful.

Thank you,
Rumen

[adrianeboyd]

Are you checking your model into a git repo at any point (which would be related to the issue you linked)?

What kind of platform/environment are you using to train and package your model? How are you copying it to the linux server?

The traceback+code might be be helpful. You can attach it as a .txt file if it gets too long.

[rumnen]

I saved the model with nlp.to_disk and only used pickle when looking for alternatives ways to transfer it.

[adrianeboyd]

I suspect the problem is related to how it's being copied. How are you copying it?

[rumnen]

Copy and paste of the entire folder containing the model or I use FileZilla when uploading to server via ftp again the entire folder.

[adrianeboyd]

Can you try packaging the model with spacy package, copying the .tar.gz or .whl file from the dist/ directory, and unpacking or installing that file on the server instead?

[adrianeboyd]

It's probably the ftp server that's modifying line endings in some files when it shouldn't, similar to the problem with git in the linked issue.