Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wheels for musl don't work #85

Open
brgl opened this issue Jun 21, 2024 · 22 comments
Open

Wheels for musl don't work #85

brgl opened this issue Jun 21, 2024 · 22 comments

Comments

@brgl
Copy link
Owner

brgl commented Jun 21, 2024

Hi @vfazio

I had some time today to test the wheels and while the ones for glibc work fine for x86-64 and aarch64, the ones linked against musl give me the following error:

Python 3.12.4 (main, Jun  6 2024, 18:26:44) [GCC 14.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gpiod
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/site-packages/gpiod/__init__.py", line 10, in <module>
    from . import _ext
ImportError: cannot import name '_ext' from partially initialized module 'gpiod' (most likely due to a circular import) (/usr/lib/python3.12/site-packages/gpiod/__init__.py)

After having installed them like that:

# pip3 install gpiod
Collecting gpiod
  Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl.metadata (3.2 kB)
Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl (96 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.2/96.2 kB 865.1 kB/s eta 0:00:00
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0

Could you advise?

@brgl
Copy link
Owner Author

brgl commented Jun 21, 2024

Please also note that this is reported by ldd:

# ldd /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so 
	ldd (0x7f9f345c7000)
	libc.musl-x86_64.so.1 => ldd (0x7f9f345c7000)
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyDict_Next: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyLong_AsSize_t: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyEval_GetGlobals: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_Free: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_CallFunction: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_Type: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyErr_SetString: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyUnicode_FromString: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyLong_FromLong: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyErr_Occurred: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyModule_AddStringConstant: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyEval_RestoreThread: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: _PyObject_New: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyArg_ParseTupleAndKeywords: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_CallFinalizerFromDealloc: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyModule_AddType: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_GetAttrString: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyList_SetItem: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyLong_AsUnsignedLong: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyModule_AddObject: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyMem_Calloc: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyMem_Free: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyTuple_GetItem: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyUnicode_AsUTF8: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyErr_SetFromErrno: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyList_GetItem: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyList_New: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_CallMethod: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyEval_SaveThread: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_GetIter: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: _Py_Dealloc: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyObject_Size: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyBool_FromLong: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyErr_NoMemory: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyModule_AddIntConstant: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyLong_FromUnsignedLong: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyModule_Create2: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyLong_AsLong: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyDict_GetItemString: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyIter_Next: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyArg_ParseTuple: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: _Py_TrueStruct: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_NotADirectoryError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: _Py_NoneStruct: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_IsADirectoryError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_BrokenPipeError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_TimeoutError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_ChildProcessError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_ValueError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_FileExistsError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_TypeError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: _Py_FalseStruct: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_FileNotFoundError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_InterruptedError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_NotImplementedError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_OSError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyExc_PermissionError: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyType_GenericNew: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyType_GenericNew: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyType_GenericNew: symbol not found
Error relocating /usr/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so: PyType_GenericNew: symbol not found

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

This is going to be fun.. I don't have an alpine x86 machine to test on. I only had alpine aarch64

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

Works on aarch64:

localhost:/tmp# python3 -m venv venv
localhost:/tmp# . venv/bin/activate
(venv) localhost:/tmp# pip install gpiod
Collecting gpiod
  Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_aarch64.whl.metadata (3.2 kB)
Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_aarch64.whl (96 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.0/97.0 kB 899.0 kB/s eta 0:00:00
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0

[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: pip install --upgrade pip
(venv) localhost:/tmp# python3
Python 3.12.3 (main, Apr 18 2024, 07:52:31) [GCC 13.2.1 20240309] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gpiod

>>> gpiod.version.__version__
'2.2.0'

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

Note that the symbols are undefined the same way between musl and glibc:

musl

(venv) localhost:/tmp# readelf --dyn-syms venv/lib/python3.12/site-packages/gpio
d/_ext.cpython-312-aarch64-linux-musl.so | grep Py | sort -k8
    83: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyArg_ParseTuple
    29: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyArg_ParseTuple[...]
    64: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyBool_FromLong
    77: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyDict_GetItemString
     3: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyDict_Next
    69: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_NoMemory
    23: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_Occurred
    50: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_SetFromErrno
    20: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_SetString
     7: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_GetGlobals
    25: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_RestoreThread
    59: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_SaveThread
    37: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_BrokenPipeError
    47: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_ChildProce[...]
    51: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_FileExistsError
    66: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_FileNotFou[...]
    68: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_Interrupte[...]
    15: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_IsADirecto[...]
    10: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_NotADirect[...]
    74: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_NotImpleme[...]
    79: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_OSError
    81: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_PermissionError
    44: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_TimeoutError
    52: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_TypeError
    49: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_ValueError
   186: 00000000000065d0   280 FUNC    GLOBAL DEFAULT    9 PyInit__ext
    82: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyIter_Next
    54: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_GetItem
    55: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_New
    34: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_SetItem
    76: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsLong
     5: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsSize_t
    36: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsUnsignedLong
    22: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_FromLong
    71: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_FromUnsig[...]
    40: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyMem_Calloc
    41: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyMem_Free
    70: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddIntC[...]
    39: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddObject
    24: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddStri[...]
    31: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddType
    72: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_Create2
    30: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallFin[...]
    12: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallFunction
    58: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallMethod
     9: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Free
    33: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttr[...]
    60: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetIter
    63: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Size
    17: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Type
    42: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyTuple_GetItem
    67: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyType_GenericNew
    43: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyUnicode_AsUTF8
    21: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyUnicode_FromString
   145: 0000000000005e90    52 FUNC    GLOBAL DEFAULT    9 Py_gpiod_GetGlob[...]
   120: 00000000000062d0   156 FUNC    GLOBAL DEFAULT    9 Py_gpiod_LineCon[...]
   147: 00000000000064b4   156 FUNC    GLOBAL DEFAULT    9 Py_gpiod_LineSet[...]
   193: 00000000000070a0   260 FUNC    GLOBAL DEFAULT    9 Py_gpiod_MakeReq[...]
   209: 0000000000005ec4    96 FUNC    GLOBAL DEFAULT    9 Py_gpiod_PyLongA[...]
   190: 0000000000005d44   328 FUNC    GLOBAL DEFAULT    9 Py_gpiod_SetErrF[...]
   172: 0000000000005d10    52 FUNC    GLOBAL DEFAULT    9 Py_gpiod_dealloc
    27: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _PyObject_New
    62: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_Dealloc
    53: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_FalseStruct
    14: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_NoneStruct
     8: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_TrueStruct

libc

(venv) vfazio@vfazio4 /tmp/tmp.pBsqfbBKHz $ readelf --dyn-syms venv/lib/python3.10/site-packages/gpiod/_ext.cpython-310-x86_64-linux-gnu.so | grep Py | sort -k8
    60: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyArg_ParseTuple
     5: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyArg_ParseTuple[...]
    39: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyBool_FromLong
    56: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyDict_GetItemString
    64: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyDict_Next
    19: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_NoMemory
    50: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_Occurred
    15: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_SetFromErrno
    22: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyErr_SetString
    55: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_GetGlobals
    13: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_RestoreThread
    73: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyEval_SaveThread
    74: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_BrokenPipeError
    76: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_ChildProce[...]
    77: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_FileExistsError
    61: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_FileNotFou[...]
    69: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_Interrupte[...]
    37: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_IsADirecto[...]
     7: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_NotADirect[...]
     6: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_NotImpleme[...]
    41: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_OSError
     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_PermissionError
    45: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_TimeoutError
    25: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_TypeError
    24: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyExc_ValueError
   181: 0000000000006a10   292 FUNC    GLOBAL DEFAULT   11 PyInit__ext
    20: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyIter_Next
    80: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_GetItem
     1: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_New
    54: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyList_SetItem
    52: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsLong
    46: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsSize_t
    63: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_AsUnsignedLong
    35: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_FromLong
    65: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyLong_FromUnsig[...]
    79: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyMem_Calloc
    14: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyMem_Free
     4: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddIntC[...]
    75: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddObject
    71: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddStri[...]
    49: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_AddType
    51: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyModule_Create2
    31: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallFin[...]
    44: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallFunction
    12: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_CallMethod
    32: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Free
    16: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetAttr[...]
    72: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_GetIter
    57: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Size
    17: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyObject_Type
    27: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyTuple_GetItem
    26: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyType_GenericNew
    66: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyUnicode_AsUTF8
    70: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND PyUnicode_FromString
   138: 00000000000063a0    36 FUNC    GLOBAL DEFAULT   11 Py_gpiod_GetGlob[...]
   115: 00000000000067b0   110 FUNC    GLOBAL DEFAULT   11 Py_gpiod_LineCon[...]
   143: 0000000000006960   110 FUNC    GLOBAL DEFAULT   11 Py_gpiod_LineSet[...]
   189: 0000000000007380   258 FUNC    GLOBAL DEFAULT   11 Py_gpiod_MakeReq[...]
   204: 00000000000063d0    70 FUNC    GLOBAL DEFAULT   11 Py_gpiod_PyLongA[...]
   184: 0000000000006280   287 FUNC    GLOBAL DEFAULT   11 Py_gpiod_SetErrF[...]
   167: 0000000000006250    34 FUNC    GLOBAL DEFAULT   11 Py_gpiod_dealloc
    68: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _PyObject_New
    18: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_Dealloc
    42: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_FalseStruct
    59: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_NoneStruct
    62: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _Py_TrueStruct


@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

If I add libpython3 as a dependency, the unfound symbol errors go away:

(venv) localhost:/tmp# patchelf --add-needed libpython3.so venv/lib/python3.12/s
ite-packages/gpiod/_ext.cpython-312-aarch64-linux-musl.so
(venv) localhost:/tmp# ldd venv/lib/python3.12/site-packages/gpiod/_ext.cpython-
312-aarch64-linux-musl.so
        /lib/ld-musl-aarch64.so.1 (0x7f8b536000)
        libpython3.so => /usr/lib/libpython3.so (0x7f8b4e3000)
        libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0x7f8b536000)
        libpython3.12.so.1.0 => /usr/lib/libpython3.12.so.1.0 (0x7f8aeb5000)

I see we're using Python.h, but it doesn't look like we're linking against libpython. I don't know if this is actually a functional issue. the package does seem to work without the linkage w/ glibc and at least musl aarch64...

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

@brgl Can you try installing gpiod in a virtualenv?

cd `mktemp -d`
python3 -m venv venv
. venv/bin/activate
pip install gpiod==2.2.0
python3 -c "import gpiod;print(gpiod.__version__)"

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

Apparently, it is recommended to not link against libpython.

https://peps.python.org/pep-0513/#libpythonx-y-so-1
pypa/manylinux#69

So, what we're doing is appropriate.

@brgl
Copy link
Owner Author

brgl commented Jun 21, 2024

@brgl Can you try installing gpiod in a virtualenv?

cd `mktemp -d`
python3 -m venv venv
. venv/bin/activate
pip install gpiod==2.2.0
python3 -c "import gpiod;print(gpiod.__version__)"

I can (on Monday though) but shouldn't it work outside venv anyway? Also: is virtualenv and venv the same thing? Yocto only seems to support virtualenv.

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

@brgl Can you try installing gpiod in a virtualenv?

cd `mktemp -d`
python3 -m venv venv
. venv/bin/activate
pip install gpiod==2.2.0
python3 -c "import gpiod;print(gpiod.__version__)"

I can (on Monday though) but shouldn't it work outside venv anyway? Also: is virtualenv and venv the same thing? Yocto only seems to support virtualenv.

They should do the same thing, but the command would be python -m virtualenv venv

@vfazio
Copy link
Contributor

vfazio commented Jun 21, 2024

Oof, Yocto. You can also try installing from the sdist and not the wheel, but you'll need the required headers on the machine for that to work.

(venv) localhost:/tmp# pip install gpiod --no-binary :all:
Collecting gpiod
  Using cached gpiod-2.2.0.tar.gz (53 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: gpiod
  Building wheel for gpiod (pyproject.toml) ... done
  Created wheel for gpiod: filename=gpiod-2.2.0-cp312-cp312-linux_aarch64.whl size=96664 sha256=e1a3e3cad1368b0c8527bf5aa51e60c708007336bd00988f771d63809a50defc
  Stored in directory: /root/.cache/pip/wheels/d9/75/8e/e28da5ee30741bb78ec7cfd2de1a0de79027cccc3d21f518f1
Successfully built gpiod
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0

[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: pip install --upgrade pip
(venv) localhost:/tmp# python3 -c "import gpiod; print(gpiod.__version__)"
2.2.0

@brgl
Copy link
Owner Author

brgl commented Jul 1, 2024

I know I can install from sdist, that's not an issue. We've had a recipe for yocto for a long time. I'm only using yocto as a minimal system with just python3 and pip to see if the wheels work fine.

@brgl
Copy link
Owner Author

brgl commented Jul 1, 2024

Here's with virtualenv:

root@qemux86-64:/tmp# python3 -m virtualenv venv
created virtual environment CPython3.12.4.final.0-64 in 146ms
  creator CPython3Posix(dest=/tmp/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==24.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
root@qemux86-64:/tmp# . ./venv/bin/activate
(venv) root@qemux86-64:/tmp# pip3 install gpiod
Collecting gpiod
  Using cached gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl.metadata (3.2 kB)
Using cached gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl (96 kB)
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0

[notice] A new release of pip is available: 24.1 -> 24.1.1
[notice] To update, run: pip install --upgrade pip
(venv) root@qemux86-64:/tmp# python3 -c "import gpiod;print(gpiod.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/venv/lib/python3.12/site-packages/gpiod/__init__.py", line 10, in <module>
    from . import _ext
ImportError: cannot import name '_ext' from partially initialized module 'gpiod' (most likely due to a circular import) (/tmp/venv/lib/python3.12)
(venv) root@qemux86-64:/tmp# 

@vfazio
Copy link
Contributor

vfazio commented Jul 1, 2024

I tested the wheels in an Alpine container, both 3.14 and 3.20, and they seemed to work fine there, which seems to imply either some python distribution nuance for Yocto or some c level library issue preventing the library from being loaded. I guess I could try to debug this at some point if i could get a yocto image with a toolchain and gdb in it

Alpine 3.14:

vfazio@vfazio4 /tmp/tmp.pBsqfbBKHz $ docker run -it --rm alpine:3.14 sh
/ # apk add python3 py3-pip py3-setuptools
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/38) Installing libbz2 (1.0.8-r1)
(2/38) Installing expat (2.5.0-r0)
(3/38) Installing libffi (3.3-r2)
(4/38) Installing gdbm (1.19-r0)
(5/38) Installing xz-libs (5.2.5-r1)
(6/38) Installing libgcc (10.3.1_git20210424-r2)
(7/38) Installing libstdc++ (10.3.1_git20210424-r2)
(8/38) Installing mpdecimal (2.5.1-r1)
(9/38) Installing ncurses-terminfo-base (6.2_p20210612-r1)
(10/38) Installing ncurses-libs (6.2_p20210612-r1)
(11/38) Installing readline (8.1.0-r0)
(12/38) Installing sqlite-libs (3.35.5-r0)
(13/38) Installing python3 (3.9.17-r0)
(14/38) Installing py3-appdirs (1.4.4-r2)
(15/38) Installing py3-chardet (4.0.0-r2)
(16/38) Installing py3-idna (3.2-r0)
(17/38) Installing py3-urllib3 (1.26.5-r0)
(18/38) Installing py3-certifi (2020.12.5-r1)
(19/38) Installing py3-requests (2.25.1-r4)
(20/38) Installing py3-msgpack (1.0.2-r1)
(21/38) Installing py3-lockfile (0.12.2-r4)
(22/38) Installing py3-cachecontrol (0.12.6-r1)
(23/38) Installing py3-colorama (0.4.4-r1)
(24/38) Installing py3-contextlib2 (0.6.0-r1)
(25/38) Installing py3-distlib (0.3.1-r3)
(26/38) Installing py3-distro (1.5.0-r3)
(27/38) Installing py3-six (1.15.0-r1)
(28/38) Installing py3-webencodings (0.5.1-r4)
(29/38) Installing py3-html5lib (1.1-r1)
(30/38) Installing py3-parsing (2.4.7-r2)
(31/38) Installing py3-packaging (20.9-r1)
(32/38) Installing py3-toml (0.10.2-r2)
(33/38) Installing py3-pep517 (0.10.0-r2)
(34/38) Installing py3-progress (1.5-r2)
(35/38) Installing py3-retrying (1.3.3-r1)
(36/38) Installing py3-ordered-set (4.0.2-r1)
(37/38) Installing py3-setuptools (52.0.0-r3)
(38/38) Installing py3-pip (20.3.4-r1)
Executing busybox-1.33.1-r8.trigger
OK: 78 MiB in 52 packages
/ # pip install -U pip
Requirement already satisfied: pip in /usr/lib/python3.9/site-packages (20.3.4)
Collecting pip
  Downloading pip-24.1.1-py3-none-any.whl (1.8 MB)
     |████████████████████████████████| 1.8 MB 3.3 MB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.3.4
    Uninstalling pip-20.3.4:
      Successfully uninstalled pip-20.3.4
Successfully installed pip-24.1.1

/ # pip install gpiod
Collecting gpiod
  Downloading gpiod-2.2.0-cp39-cp39-musllinux_1_2_x86_64.whl.metadata (3.2 kB)
Downloading gpiod-2.2.0-cp39-cp39-musllinux_1_2_x86_64.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 kB 2.6 MB/s eta 0:00:00
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
/ # python3 -vv -c "import gpiod" 2>&1 | grep gpiod
# trying /gpiod.cpython-39-x86_64-linux-musl.so
# trying /gpiod.cpython-39-x86_64-linux-gnu.so
# trying /gpiod.abi3.so
# trying /gpiod.so
# trying /gpiod.py
# trying /gpiod.pyc
# trying /usr/lib/python3.9/gpiod.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/gpiod.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/gpiod.abi3.so
# trying /usr/lib/python3.9/gpiod.so
# trying /usr/lib/python3.9/gpiod.py
# trying /usr/lib/python3.9/gpiod.pyc
# trying /usr/lib/python3.9/lib-dynload/gpiod.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/lib-dynload/gpiod.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/lib-dynload/gpiod.abi3.so
# trying /usr/lib/python3.9/lib-dynload/gpiod.so
# trying /usr/lib/python3.9/lib-dynload/gpiod.py
# trying /usr/lib/python3.9/lib-dynload/gpiod.pyc
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/__init__.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/__init__.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/__init__.cpython-39.pyc'
# trying /usr/lib/python3.9/site-packages/gpiod/_ext.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/_ext.cpython-39-x86_64-linux-gnu.so
# extension module 'gpiod._ext' loaded from '/usr/lib/python3.9/site-packages/gpiod/_ext.cpython-39-x86_64-linux-gnu.so'
# extension module 'gpiod._ext' executed from '/usr/lib/python3.9/site-packages/gpiod/_ext.cpython-39-x86_64-linux-gnu.so'
import 'gpiod._ext' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7c11361267f0>
# trying /usr/lib/python3.9/site-packages/gpiod/line.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/line.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/line.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/line.so
# trying /usr/lib/python3.9/site-packages/gpiod/line.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/line.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/line.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/line.cpython-39.pyc'
import 'gpiod.line' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1136126670>
# trying /usr/lib/python3.9/site-packages/gpiod/chip.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/chip.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/chip.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/chip.cpython-39.pyc'
# trying /usr/lib/python3.9/site-packages/gpiod/chip_info.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip_info.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip_info.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip_info.so
# trying /usr/lib/python3.9/site-packages/gpiod/chip_info.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/chip_info.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/chip_info.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/chip_info.cpython-39.pyc'
import 'gpiod.chip_info' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1136153be0>
# trying /usr/lib/python3.9/site-packages/gpiod/exception.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/exception.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/exception.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/exception.so
# trying /usr/lib/python3.9/site-packages/gpiod/exception.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/exception.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/exception.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/exception.cpython-39.pyc'
import 'gpiod.exception' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1136153d60>
# trying /usr/lib/python3.9/site-packages/gpiod/info_event.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/info_event.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/info_event.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/info_event.so
# trying /usr/lib/python3.9/site-packages/gpiod/info_event.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/info_event.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/info_event.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/info_event.cpython-39.pyc'
# trying /usr/lib/python3.9/site-packages/gpiod/line_info.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_info.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_info.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_info.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_info.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/line_info.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/line_info.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/line_info.cpython-39.pyc'
import 'gpiod.line_info' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135f27a60>
import 'gpiod.info_event' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135f19790>
# trying /usr/lib/python3.9/site-packages/gpiod/internal.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/internal.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/internal.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/internal.so
# trying /usr/lib/python3.9/site-packages/gpiod/internal.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/internal.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/internal.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/internal.cpython-39.pyc'
import 'gpiod.internal' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135e4fa90>
# trying /usr/lib/python3.9/site-packages/gpiod/line_settings.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_settings.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_settings.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_settings.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_settings.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/line_settings.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/line_settings.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/line_settings.cpython-39.pyc'
import 'gpiod.line_settings' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135e64df0>
# trying /usr/lib/python3.9/site-packages/gpiod/line_request.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_request.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_request.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_request.so
# trying /usr/lib/python3.9/site-packages/gpiod/line_request.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/line_request.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/line_request.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/line_request.cpython-39.pyc'
# trying /usr/lib/python3.9/site-packages/gpiod/edge_event.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/edge_event.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/edge_event.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/edge_event.so
# trying /usr/lib/python3.9/site-packages/gpiod/edge_event.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/edge_event.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/edge_event.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/edge_event.cpython-39.pyc'
import 'gpiod.edge_event' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135e72e50>
import 'gpiod.line_request' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135e64970>
import 'gpiod.chip' # <_frozen_importlib_external.SourceFileLoader object at 0x7c11361264c0>
# trying /usr/lib/python3.9/site-packages/gpiod/version.cpython-39-x86_64-linux-musl.so
# trying /usr/lib/python3.9/site-packages/gpiod/version.cpython-39-x86_64-linux-gnu.so
# trying /usr/lib/python3.9/site-packages/gpiod/version.abi3.so
# trying /usr/lib/python3.9/site-packages/gpiod/version.so
# trying /usr/lib/python3.9/site-packages/gpiod/version.py
# /usr/lib/python3.9/site-packages/gpiod/__pycache__/version.cpython-39.pyc matches /usr/lib/python3.9/site-packages/gpiod/version.py
# code object from '/usr/lib/python3.9/site-packages/gpiod/__pycache__/version.cpython-39.pyc'
import 'gpiod.version' # <_frozen_importlib_external.SourceFileLoader object at 0x7c1135e83100>
import 'gpiod' # <_frozen_importlib_external.SourceFileLoader object at 0x7c11361aaac0>
# cleanup[2] removing gpiod._ext
# cleanup[2] removing gpiod.line
# cleanup[2] removing gpiod.chip_info
# cleanup[2] removing gpiod.exception
# cleanup[2] removing gpiod.line_info
# cleanup[2] removing gpiod.info_event
# cleanup[2] removing gpiod.internal
# cleanup[2] removing gpiod.line_settings
# cleanup[2] removing gpiod.edge_event
# cleanup[2] removing gpiod.line_request
# cleanup[2] removing gpiod.chip
# cleanup[2] removing gpiod.version
# cleanup[2] removing gpiod
# destroy gpiod
# destroy gpiod.line
# destroy gpiod.chip_info
# destroy gpiod.exception
# destroy gpiod.line_info
# destroy gpiod.info_event
# destroy gpiod.internal
# destroy gpiod.line_settings
# destroy gpiod.edge_event
# destroy gpiod.line_request
# destroy gpiod.chip
# destroy gpiod.version
# cleanup[3] wiping gpiod._ext
# destroy gpiod._ext

Alpine 3.20:

/tmp/tmp.fnnEjD # python3 -m venv venv
/tmp/tmp.fnnEjD # . venv/bin/activate
(venv) /tmp/tmp.fnnEjD # pip install gpiod
Collecting gpiod
  Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl.metadata (3.2 kB)
Downloading gpiod-2.2.0-cp312-cp312-musllinux_1_2_x86_64.whl (96 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.2/96.2 kB 2.7 MB/s eta 0:00:00
Installing collected packages: gpiod
Successfully installed gpiod-2.2.0

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: pip install --upgrade pip
(venv) /tmp/tmp.fnnEjD # python3 -vv -c "import gpiod" 2>&1 | grep gpiod
# trying /tmp/tmp.fnnEjD/gpiod.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/gpiod.abi3.so
# trying /tmp/tmp.fnnEjD/gpiod.so
# trying /tmp/tmp.fnnEjD/gpiod.py
# trying /tmp/tmp.fnnEjD/gpiod.pyc
# trying /usr/lib/python3.12/gpiod.cpython-312-x86_64-linux-musl.so
# trying /usr/lib/python3.12/gpiod.abi3.so
# trying /usr/lib/python3.12/gpiod.so
# trying /usr/lib/python3.12/gpiod.py
# trying /usr/lib/python3.12/gpiod.pyc
# trying /usr/lib/python3.12/lib-dynload/gpiod.cpython-312-x86_64-linux-musl.so
# trying /usr/lib/python3.12/lib-dynload/gpiod.abi3.so
# trying /usr/lib/python3.12/lib-dynload/gpiod.so
# trying /usr/lib/python3.12/lib-dynload/gpiod.py
# trying /usr/lib/python3.12/lib-dynload/gpiod.pyc
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/__init__.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__init__.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/__init__.cpython-312.pyc'
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so
# extension module 'gpiod._ext' loaded from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so'
# extension module 'gpiod._ext' executed from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/_ext.cpython-312-x86_64-linux-musl.so'
import 'gpiod._ext' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f5db3d315b0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line.cpython-312.pyc'
import 'gpiod.line' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3d317f0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/chip.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/chip.cpython-312.pyc'
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip_info.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip_info.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip_info.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip_info.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/chip_info.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/chip_info.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/chip_info.cpython-312.pyc'
import 'gpiod.chip_info' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3daaea0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/exception.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/exception.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/exception.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/exception.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/exception.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/exception.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/exception.cpython-312.pyc'
import 'gpiod.exception' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3daaed0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/info_event.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/info_event.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/info_event.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/info_event.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/info_event.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/info_event.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/info_event.cpython-312.pyc'
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_info.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_info.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_info.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_info.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_info.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_info.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_info.cpython-312.pyc'
import 'gpiod.line_info' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3bf87d0>
import 'gpiod.info_event' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3bf85f0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/internal.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/internal.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/internal.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/internal.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/internal.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/internal.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/internal.cpython-312.pyc'
import 'gpiod.internal' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3bf8fe0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_settings.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_settings.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_settings.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_settings.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_settings.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_settings.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_settings.cpython-312.pyc'
import 'gpiod.line_settings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3c1c920>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_request.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_request.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_request.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_request.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_request.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/line_request.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/line_request.cpython-312.pyc'
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/edge_event.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/edge_event.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/edge_event.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/edge_event.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/edge_event.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/edge_event.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/edge_event.cpython-312.pyc'
import 'gpiod.edge_event' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db39b6330>
import 'gpiod.line_request' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3c1cd40>
import 'gpiod.chip' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3d7cfb0>
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/version.cpython-312-x86_64-linux-musl.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/version.abi3.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/version.so
# trying /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/version.py
# /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/version.cpython-312.pyc matches /tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/version.py
# code object from '/tmp/tmp.fnnEjD/venv/lib/python3.12/site-packages/gpiod/__pycache__/version.cpython-312.pyc'
import 'gpiod.version' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db39b7a40>
import 'gpiod' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5db3d30f20>
# cleanup[2] removing gpiod._ext
# cleanup[2] removing gpiod.line
# cleanup[2] removing gpiod.chip_info
# cleanup[2] removing gpiod.exception
# cleanup[2] removing gpiod.line_info
# cleanup[2] removing gpiod.info_event
# cleanup[2] removing gpiod.internal
# cleanup[2] removing gpiod.line_settings
# cleanup[2] removing gpiod.edge_event
# cleanup[2] removing gpiod.line_request
# cleanup[2] removing gpiod.chip
# cleanup[2] removing gpiod.version
# cleanup[2] removing gpiod
# destroy gpiod
# destroy gpiod.line
# destroy gpiod.chip_info
# destroy gpiod.exception
# destroy gpiod.line_info
# destroy gpiod.info_event
# destroy gpiod.internal
# destroy gpiod.line_settings
# destroy gpiod.edge_event
# destroy gpiod.line_request
# destroy gpiod.chip
# destroy gpiod.version
# cleanup[3] wiping gpiod._ext
# destroy gpiod._ext

The older version of alpine looks for a lib called "gnu", this is OK because installs were quirked due the ABI not having been broken out correctly in the past. This is not (or shouldn't be) an issue in newer versions of Python.

Note in newer versions of some distros (like alpine) you cannot globally install a package via pip as those are intended to be distro provided packages (via apk or apt or yum), which is why i asked to try this in a virtualenv to make sure this wasn't at play in any way.

/ # pip install gpiod
error: externally-managed-environment

× This environment is externally managed
╰─>
    The system-wide python installation should be maintained using the system
    package manager (apk) only.

    If the package in question is not packaged already (and hence installable via
    "apk add py3-somepackage"), please consider installing it inside a virtual
    environment, e.g.:

    python3 -m venv /path/to/venv
    . /path/to/venv/bin/activate
    pip install mypackage

    To exit the virtual environment, run:

    deactivate

    The virtual environment is not deleted, and can be re-entered by re-sourcing
    the activate file.

    To automatically manage virtual environments, consider using pipx (from the
    pipx package).

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

@brgl
Copy link
Owner Author

brgl commented Jul 1, 2024

Ok, I guess this isn't that much of an issue as yocto already has a recipe for python bindings but it would be worth to figure out why this is happening at some point. Let's leave it for now.

@vfazio
Copy link
Contributor

vfazio commented Jul 1, 2024

Ok, we can leave this open with a plan to investigate further at some point

@pdcastro
Copy link

@brgl, if you pip install gpiod with the --no-binary gpiod option (which forces compilation from source), do you still get the same ImportError?

# pip3 install --no-binary gpiod gpiod
...
# python3 -c "import gpiod;print(gpiod.__version__)"
2.2.1

@vfazio
Copy link
Contributor

vfazio commented Oct 10, 2024

That's essentially how Yocto's recipe works, by manually building/installing the wheel from an sdist AFAIK.

My gut says it's a problem with either Python search paths or a libc incompatibility since Yocto builds all of that from scratch.

It would be interesting to see just the output from python3 -vvv -c "import gpiod" to see the search paths it's attempting. The last time i saw this was actually as part of a build and it was because the external library wasn't in the correct location.

@brgl
Copy link
Owner Author

brgl commented Oct 10, 2024

@brgl, if you pip install gpiod with the --no-binary gpiod option (which forces compilation from source), do you still get the same ImportError?

# pip3 install --no-binary gpiod gpiod
...
# python3 -c "import gpiod;print(gpiod.__version__)"
2.2.1

Sure but what good is the wheel then? :)

@vfazio
Copy link
Contributor

vfazio commented Oct 10, 2024

I just noticed you did this in qemu. If you have a yocto config you can share that i can use to reproduce, ill try to give it a shot when i get back from Japan in Nov.

@brgl
Copy link
Owner Author

brgl commented Oct 10, 2024

It's literally the simplest core-image-minimal with python3-pip in IMAGE_INSTALL.

Full config is:

MACHINE = "qemux86-64"
DISTRO = "poky"

EXTRA_IMAGE_FEATURES = "debug-tweaks"

INIT_MANAGER = "sysvinit"

IMAGE_INSTALL:append = " \
    python3-pip \
    connman \
"

TCLIBC = "musl"

@pdcastro
Copy link

Sure but what good is the wheel then? :)

If the ImportError also happens with pip3 install --no-binary gpiod gpiod, this rules out the issue being with the wheel published to PyPI (because the wheel published to PyPI is not used). I didn’t mean that’s a solution to problem, just a debugging step to isolate the problem.

@brgl
Copy link
Owner Author

brgl commented Oct 10, 2024

Sure but what good is the wheel then? :)

If the ImportError also happens with pip3 install --no-binary gpiod gpiod, this rules out the issue being with the wheel published to PyPI (because the wheel published to PyPI is not used). I didn’t mean that’s a solution to problem, just a debugging step to isolate the problem.

Ok, so I was being unclear, I meant "sure I can test it" not "sure, it fails too" because I didn't test it. I can do it tomorrow but that would be a whole different environment if I pull in a slew of development tools. And I'm pretty sure it will work then, because just pulling in python3-gpiod from the recipe works fine on musl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants