Build Python 3.13 wheels (not free-threaded)

[edgarrmondragon]

I can close this if #703 gets more attention, but I've confirmed changes here allow the wheels to be built in 3.8-3.13, and all tests are passing.

This warning is currently raised, but fixing it might be beyond the scope of this PR:

PytestUnraisableExceptionWarning: Exception ignored in PyDict_GetItem(); consider using PyDict_GetItemRef() or PyDict_GetItemWithError(): None

python/cpython#106004

• Closes 3.13 support #698
• Closes Doesn't install on python 3.13 #741

[edgarrmondragon]

I think this is ready for an initial review.

Mostly followed advice from

• python/cpython#121489 (comment)
• python/cpython#121125 (comment)

[edgarrmondragon]

This missing parameter is causing a ton of warnings like this:

msgspec/_utils.py:130: DeprecationWarning: Failing to pass a value to the 'type_params' parameter of 'typing._eval_type' is deprecated, as it leads to incorrect behaviour when calling typing._eval_type on a stringified annotation that references a PEP 695 type parameter. It will be disallowed in Python 3.15.

python/cpython#118418

[edgarrmondragon]

This warning is currently raised but fixing it might be beyond the scope of this PR to fix it:

PytestUnraisableExceptionWarning: Exception ignored in PyDict_GetItem(); consider using PyDict_GetItemRef() or PyDict_GetItemWithError(): None

FWIW I think this is hiding an actual error:

PytestUnraisableExceptionWarning: Exception ignored in PyDict_GetItem(); consider using PyDict_GetItemRef() or PyDict_GetItemWithError(): None
    
    Traceback (most recent call last):
      File "/opt/_internal/cpython-3.13.0b3/lib/python3.13/typing.py", line 2251, in __hash__
        return hash((self.__origin__, self.__metadata__))
    TypeError: unhashable type: 'set'
    
      warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

in

msgspec/tests/test_inspect.py

Lines 790 to 793 in 2c37da0

	def test_inspect_with_unhashable_metadata():
	typ = Annotated[int, {"unhashable"}]
	assert mi.type_info(typ) == mi.IntType()

[edgarrmondragon]

macos-11 runners aren't available anymore, so I updated it to macos-14 in 748f4bb.

https://github.blog/changelog/2024-05-20-actions-upcoming-changes-to-github-hosted-macos-runners/#macos-11-deprecation-and-removal

[edgarrmondragon]

macos-11 runners aren't available anymore, so I updated it to macos-14 in 748f4bb.

https://github.blog/changelog/2024-05-20-actions-upcoming-changes-to-github-hosted-macos-runners/#macos-11-deprecation-and-removal

With that change, the wheels are finally built instead of the workflow getting stuck forever:

 12 wheels produced in 5 minutes:
  msgspec-0+untagged.1.g766a8f7-cp310-cp310-macosx_10_9_x86_64.whl    186 kB
  msgspec-0+untagged.1.g766a8f7-cp310-cp310-macosx_11_0_arm64.whl     180 kB
  msgspec-0+untagged.1.g766a8f7-cp311-cp311-macosx_10_9_x86_64.whl    184 kB
  msgspec-0+untagged.1.g766a8f7-cp311-cp311-macosx_11_0_arm64.whl     178 kB
  msgspec-0+untagged.1.g766a8f7-cp312-cp312-macosx_10_9_x86_64.whl    186 kB
  msgspec-0+untagged.1.g766a8f7-cp312-cp312-macosx_11_0_arm64.whl     180 kB
  msgspec-0+untagged.1.g766a8f7-cp313-cp313-macosx_10_13_x86_64.whl   186 kB
  msgspec-0+untagged.1.g766a8f7-cp313-cp313-macosx_11_0_arm64.whl     180 kB
  msgspec-0+untagged.1.g766a8f7-cp38-cp38-macosx_10_9_x86_64.whl      186 kB
  msgspec-0+untagged.1.g766a8f7-cp38-cp38-macosx_11_0_arm64.whl       180 kB
  msgspec-0+untagged.1.g766a8f7-cp39-cp39-macosx_10_9_x86_64.whl      186 kB
  msgspec-0+untagged.1.g766a8f7-cp39-cp39-macosx_11_0_arm64.whl       180 kB

https://github.com/edgarrmondragon/msgspec/actions/runs/10030128227/job/27719084757

[tacaswell]

I can also get msgspec to build with py313 with this branch, but not the branch in #703

[edgarrmondragon]

@jcrist This is ready for review at your convenience :)

[edgarrmondragon]

Updated to use Python 3.13.0rc1

[edgarrmondragon]

Updated to use Python 3.13.0rc2

[ofek]

@jcrist Can you please run the CI?

[Tishka17]

Python 3.13 release is scheduled for then upcoming Monday. Are there any plans to merge this PR and publish a new version?

[Tishka17]

Python 3.13 is out there.

[maurerle]

This closes #741 and #698

I was building on debian 13 trixie which uses GCC 14, I therefore had a few more required adjustments on top of yours:

maurerle@2aa5db5

To build this locally, I used conda:

conda create -n py313 python-freethreading -c conda-forge
conda activate pyt313
git clone -b py313 [email protected]:maurerle/msgspec.git
cd msgspec
pip install .

[maurerle]

I do get a segfault though when importing msgspec.. Even with GIL enabled

(py313) ➜  msgspec git:(py313) ✗ python -X gil=1 -X faulthandler
Python 3.13.0 experimental free-threading build | packaged by conda-forge | (main, Oct  8 2024, 20:16:19) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import msgspec
Fatal Python error: Segmentation fault

Current thread 0x00007fcd5e8ba640 (most recent call first):
  File "~/github/msgspec/msgspec/__init__.py", line 19 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1022 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<python-input-1>", line 1 in <module>

[edgarrmondragon]

I do get a segfault though when importing msgspec.. Even with GIL enabled

(py313) ➜  msgspec git:(py313) ✗ python -X gil=1 -X faulthandler
Python 3.13.0 experimental free-threading build | packaged by conda-forge | (main, Oct  8 2024, 20:16:19) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import msgspec
Fatal Python error: Segmentation fault

Current thread 0x00007fcd5e8ba640 (most recent call first):
  File "~/github/msgspec/msgspec/__init__.py", line 19 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1022 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<python-input-1>", line 1 in <module>

@maurerle thanks for finding that. I'll give a read to https://py-free-threading.github.io/porting/, unless someone beats me to it and knows what's wrong here.

[maurerle]

When executing this on a gil build (e.g. conda create -n py313-gil python=3.13)
One gets an error if executed from the build directory:

>>> import msgspec
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import msgspec
  File "~/github/msgspec/msgspec/__init__.py", line 1, in <module>
    from ._core import (
    ...<14 lines>...
    )
ModuleNotFoundError: No module named 'msgspec._core'

So make sure to execute this not in the build directory :)
It works fine then :)
But I don't have a clue how to solve the above segfault either.

The segfault also occurs when starting the above mentioned freethreading build with an active GIL as in
python -X gil=1 -c "import msgspec"

[jcrist]

Thanks for the PR (and everyone for their patience). With a new kid and recent layoff, I've been a bit busy and msgspec maintenance has languished. I really appreciate the help and patience here.

I have a few small fixups I want to land with this release, but I'd expect a release supporting Python 3.13 (not with freethreading) out in the next day or two. I'll work on adding freethreading support after getting that initial release out.

[ekneg54]

Hi,

with this merged I thought that I should be able to install msgspec on python 3.13. But it turns out, that this is not the case as you could see:

I wonder if I am missing something. Could you help?

[maurerle]

You are missing the python3-dev dependency.
Install it using your system package manager

[mj0nez]

It was merged but not released or am I mistaken? An install from main should work for sure: install from vcs

[ekneg54]

thank you. I wait for the release. Do you know when it is planned?

[mj0nez]

No, but soon I guess.

[st3fan]

@jcrist Is there anything the community can do to help to get a release out? There is probably some collective spare hacking time available during the break now. Let us know if you need help testing or writing release notes, etc.