[BUG] Path separator in Python include path breaks building extension modules

[manueljacob]

When the Python include dir contains a colon (as e.g. in /home/user/colon_in_path_bug/foo:bar/include), extension modules get compiled with the wrong include dirs being passed to the C compiler (e.g. gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/home/user/colon_in_path_bug/foo -Ibar/include/python3.15 -c foo.c -o build/temp.linux-x86_64-cpython-315/foo.o).

This bug was introduced by python/cpython@85e102a#diff-cc924b02acb3e0db7d519de53c31e0b3945e013e7d387439d8ea32355a1ae99eR164. It was required to return multiple include dirs from distutils.sysconfig.get_python_inc(), so it was changed to join multiple include directories by a path separator. However, if the include dir e.g. returned by sysconfig.get_path("include") already contains a path separator, there is ambiguity about whether the path seperator is used as a path separator or happens to be in the path.

[manueljacob]

I think this should be fixed by letting the function returning the include paths (now setuptools._distutils.sysconfig.get_python_inc()) return a list. However, I’m not sure about the status of the API. Is it public or internal API? I see the following possibilities:

1. Change setuptools._distutils.sysconfig.get_python_inc() to return a list.
2. Introduce a new function (e.g. setuptools._distutils.sysconfig.get_python_include_dirs()) returning a list and let setuptools._distutils.sysconfig.get_python_inc() return the same result as before (joining the paths from the new function).

[abravalheri]

Hi @manueljacob, thank you very much for the report.

This may need further discussion, as it was an intentional change driven by Python core developers (see python/cpython#79480 (comment)). Note that while not strictly forbidden, the : character has historically carried special meaning in POSIX systems - which may lead to other systems/tools expecting a certain behaviour.

Since this concerns changes in the _distutils directory, it might be more appropriate to raise the issue in the pypa/distutils repository, where related fixes are typically handled.