avrdude Python binding CMake configuration improvement

[leventelist]

I have a script that builds AVR toolchain, including avrdude. The sctipt installs the artifacts to a directory in my user's home. I've noticed that when I want to install v8.0, it directly targets /usr/local for installing some python module. Is there any workaround not doing it? I run my script as a normal user, and it keeps failing, as it is not writable by a non superuser.

I came up with this patch.

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index e358e4b1..7bef99ed 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -403,8 +403,10 @@ install(FILES "avrdude.1"
 	)
 
 if(HAVE_SWIG)
-    execute_process(COMMAND ${Python3_EXECUTABLE} -c "import site; print(site.getsitepackages()[0])"
-                    OUTPUT_VARIABLE PYTHON_SITE_PACKAGES OUTPUT_STRIP_TRAILING_WHITESPACE)
+    if(NOT DEFINED PYTHON_SITE_PACKAGES)
+      execute_process(COMMAND ${Python3_EXECUTABLE} -c "import site; print(site.getsitepackages()[0])"
+                      OUTPUT_VARIABLE PYTHON_SITE_PACKAGES OUTPUT_STRIP_TRAILING_WHITESPACE)
+    endif()
     install(TARGETS swig_avrdude DESTINATION ${PYTHON_SITE_PACKAGES})
     install(FILES ${CMAKE_BINARY_DIR}/src/swig_avrdude.py DESTINATION ${PYTHON_SITE_PACKAGES})
     install(DIRECTORY python/ DESTINATION ${CMAKE_INSTALL_DATADIR}/avrdude FILES_MATCHING PATTERN "*.ui")

And call cmake with something like this: "-DPYTHON_SITE_PACKAGES=python"

[ndim]

For cases where you do not want to build the swig/python/avrdude-gui things at all, my quick hack workaround is to run

sed -i 's|^find_package(SWIG|# find_package(SWIG|' CMakeLists.txt

before running cmake for the first time.

[ndim]

I would love to see a clean solution for installing a software package into a custom PREFIX but some things needing to be installed into a well-known directory which is not inside PREFIX.

In this case, this is about python sitedir, but the same thing acould also be bash completions dir, xdg icons dir, man page dir, etc.

Some of these directories could be accommodated by setting environment variables like PYTHONPATH, MANPATH, etc., but that becomes cumbersome very quick and is not even possible for some things.

Some things can be delegated to the person installing the package to do manually (or a OS package doing, like e.g. installing udev rules), but manually installing all the python files into the respective directories certainly is too much to ask.

Anyway... looking for a nice way to handle such things. Please tell me if you know one.

[mcuee]

I think it is easier to use CMake command line option to disable the Python binding. Let me try and give an option later.

As for installation of Python bindings, I think it is a tough topic especially under Windows (eg: MSVC), based on my experiences with libftdi project. So we would say to use manual instalation for Windows and try to automatic installation for other OS.

Reference libftdi:
http://developer.intra2net.com/git/?p=libftdi

FYI, I am kind of the Windows packager for libftdi binaries for Windows for a long time (2010 to 2020). Now MSYS2 mingw makes it simpler so it is not really needed any more.
https://sourceforge.net/projects/picusb/files/

[mcuee]

I was hoping ./build.sh -f "-D HAVE_SWIG=0" will work to disable Python bindings but it does not work.

• CMAKE issues and Improvement Ideas #1920 (comment)

[mcuee]

@leventelist and @ndim

I think there is kind of a bug here that the Python binding has kind of hard-coded path which does not honor CMAKE option: -D CMAKE_INSTALL_PREFIX:PATH=/home/mcuee/bin.

mcuee@PCRyzen5600G:~/build/avr/avrdude$ ./build.sh -f "-D BUILD_DOC=1 -D HAVE_LINUXGPIO=1 -D HAVE_PARPORT=1 -D HAVE_LINUXSPI=1 -D CMAKE_INSTALL_PREFIX:PATH=/home/mcuee/bin"
-- The C compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/bin/git (found version "2.43.0") 
-- Found FLEX: /usr/bin/flex (found version "2.6.4") 
-- Found BISON: /usr/bin/bison (found version "3.8.2")
-- Looking for libelf.h
-- Looking for libelf.h - found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - not found
-- Looking for usb.h
-- Looking for usb.h - found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for hidapi/hidapi.h
-- Looking for hidapi/hidapi.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1") 
-- Checking for module 'libgpiod'...
-- Found module 'libgpiod' (found version "1.6.3")
-- Found SWIG: /usr/bin/swig4.0 (found suitable version "4.2.0", minimum required is "4.0") found components: python 
-- Found Python3: /usr/bin/python3 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed 
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DO HAVE    libhidapi
-- DON'T HAVE libftdi
-- DO HAVE    libftdi1
-- DO HAVE    libreadline
-- DO HAVE    libserialport
-- ENABLED    doc
-- ENABLED    parport
-- ENABLED    linuxgpio
-- DO HAVE    libgpiod (1.6.3)
-- ENABLED    linuxspi
-- DO HAVE    swig+Python3
-- ----------------------
-- Configuring done (0.8s)
-- Generating done (0.0s)
-- Build files have been written to: /home/mcuee/build/avr/avrdude/build_linux
... 


mcuee@PCRyzen5600G:~/build/avr/avrdude$ cmake --build build_linux --target install
[  0%] Built target conf
[ 42%] Built target libavrdude
[ 44%] Built target avrdude
[ 45%] Built target swig_avrdude_swig_compilation
[ 88%] Built target swig_avrdude
[ 88%] Built target avrdude_binaries
[ 92%] Built target info
[ 96%] Built target pdf
[100%] Built target html
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /home/mcuee/bin/bin/avrdude
-- Installing: /home/mcuee/bin/bin/elf2tag
-- Installing: /home/mcuee/bin/lib/libavrdude.a
-- Installing: /home/mcuee/bin/include/libavrdude.h
-- Installing: /home/mcuee/bin/include/libavrdude-avrintel.h
-- Installing: /home/mcuee/bin/etc/avrdude.conf
-- Installing: /home/mcuee/bin/share/man/man1/avrdude.1
-- Installing: /usr/local/lib/python3.12/dist-packages/_swig_avrdude.so
CMake Error at src/cmake_install.cmake:103 (file):
  file INSTALL cannot copy file
  "/home/mcuee/build/avr/avrdude/build_linux/src/_swig_avrdude.so" to
  "/usr/local/lib/python3.12/dist-packages/_swig_avrdude.so": Permission
  denied.
Call Stack (most recent call first):
  cmake_install.cmake:47 (include)


gmake: *** [Makefile:100: install] Error 1

[ndim]

The well known well-known directory problem.

[ndim]

Interestingly, the AX_* macros for Automake builds provide directory definitions which install python code to whatever --prefix you give it (PR coming in a few days).

[ndim]

Oh, and as the whole installation and the startup which finds the installed files etc is a bit unusual, we might be able to do something about working with a non-well-known install dir. Some changes to the cmake code will be required.

[leventelist]

@ndim The patch I included?

[ndim]

@leventelist I was thinking more along the lines of installing to /home/user/my-avr-toolset-root prefix and avrdude ending up in /home/user/my-avr-toolset-root/bin and the python stuff in /home/user/my-avr-toolset-root/lib/python3.12/site-packages, without needing to give any extra arguments to cmake.

Then we just need to install a correctly adapted avrdude-gui script which adds the proper path to PYTHONPATH.

[mcuee]

BTW, mostly Windows users using MSVC has to manually install the Python extension. I tend to believe it is the same for macOS Homebrew users.

Homebrew avrdude formula does not enable Swig/Python at all. In fact, it is even missing libserialport support.
https://github.com/Homebrew/homebrew-core/blob/ca85cae75ef9d3e0446b594f1e80bd2998da28ec/Formula/a/avrdude.rb

[mcuee]

Swig and Python detection may also need to be improved.

The following is from my Mac Mini M1 running the latest macOS 15.0 and Homebrew. It detects the wrong default Python installation.

My /usr/local is mainly for x86_64 version of Homebrew.

avrdude_test/avrdude_main - (main)> which python3
/opt/homebrew/bin/python3

avrdude_test/avrdude_main - (main)> which swig
/opt/homebrew/bin/swig

avrdude_test/avrdude_main - (main)> brew --prefix
/opt/homebrew

avrdude_test/avrdude_main - (main)> ./build.sh 
-- The C compiler identification is AppleClang 16.0.0.16000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /opt/homebrew/bin/git (found version "2.46.2")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Found BISON: /usr/bin/bison (found version "2.3")
-- Looking for libelf.h
-- Looking for libelf.h - not found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - found
-- Looking for usb.h
-- Looking for usb.h - found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for hidapi/hidapi.h
-- Looking for hidapi/hidapi.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Found SWIG: /opt/homebrew/bin/swig (found suitable version "4.2.1", minimum required is "4.0") found components: python
-- Found Python3: /usr/local/Frameworks/Python.framework/Versions/3.12/bin/python3.12 (found version "3.12.6") found components: Interpreter Development Development.Module Development.Embed
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DO HAVE    libhidapi
-- DON'T HAVE libftdi
-- DO HAVE    libftdi1
-- DO HAVE    libreadline
-- DO HAVE    libserialport
-- DISABLED   doc
-- DISABLED   parport
  -- DISABLED   linuxgpio
  -- DISABLED   linuxspi
  -- DO HAVE    swig+Python3
  -- ----------------------
  -- Configuring done (3.0s)
  -- Generating done (0.0s)
  -- Build files have been written to: /Users/mcuee/build/avr/avrdude_test/avrdude_main/build_darwin