dynamic crossenv + python3*-wheels + python310-313 updates (#6282)

* python311: Remove optional wheels (to be migrated to its own spk)

* python313: Initial commit, fails to build on readline

* python-wheels: First attempt, to be further developed

* py311: Re-enable lto

* python313: First working build

* python313: Remove pgo optimizations as non-functional in cross-builds

* python-wheels: First integration of additional wheels for testing

* mariadb-connector-c: Update from version 3.3.5 to 3.3.11 and fix build failure

* python313: No longer support armv5 arch due to c++11 support needed

* spksrc.crossenv.mk: First iteration of on-demand crossenv creation

* spksrc.crossenv.mk: Print crossenv resulting package list

* py310-311-313: Remove crossenv creation

* mk/crossenv: Remove aiohttp example as obsolete

* mk/crossenv: disable meson and scikit-build-core temporarily

* spksrc.crossenv.mk: Misc fixes

* python-module.mk: Make compatible with spksrc.crossenv.mk

* wheel.mk: Make compatible with spksrc.crossenv.mk

* crossenv.mk: Allow building from PYTHON_PACKAGE dependend spk

* py310-311: Fix lto vs pgo optimizations

* wheel.mk: Use $name-$version using - instead of == for crossenv

* python.mk: do not symlink crossenv and python-cc.mk

* python.mk: Move mandatory PYO3_* variables for rust cross-compiling

* crossenv.mk: Fix crossenv creation and pip tool calls

* crossenv/requirements-default.txt: Update to latest wheel versions

* spk.mk: Add a crossenvclean clean-up option

* python.mk + crossenv.mk: Functional versions

* python.mk: Further clean-up and removal of duplicate variables

* py311-312-312: Update pure requirements to latest versions

* py310-311-313: Further clean-up Makefile from unused variables

* py310-311: No longer need to remove test folder

* py313: Fix symlink creation

* crossenv.mk: Include missing OPENSSL_*_DIR

* spk.mk: Update crossenvclean and add crossenvcleanall

* spk.mk: crossenvclean remove cookie file

* crossenv: Fix python version and use status cookie

* crossenv-default: Enforce using msgpack==1.0.5 for py313 builds

* fix crossenv installation by packages using prebuilt python

- spksrc.crossenv.mk: SPK_NAME and PYTHON_PACKAGE_WORK_DIR must be defined when PYTHON_PACKAGE is used

* fix python311 for qoriq

* spksrc.python-wheel.mk: fix include of python-cc.mk

* rdiff-backup: add crossenv for frozenlist

* rdiff-backup: add crossenv for yarl

* fix for version less crossenv requirements

- use dash (-) as version separator
- rdiff-backup: version less requirements for frozenlist and yarl
- adjust logging

* python.mk: revert support for ADDITIONAL_CROSSENV_WHEELS

- obsolete with wheel specific crossenv

* Revert "borgbackup: Package bump to force build"

This reverts commit e95af06.

* use python -m build to create wheels with python-wheel.mk

- add build to default crossenv
- fix python-wheel.mk (include spksrc.crossenv.mk)
- use workaround to access crossenv to build wheels with python-wheel.mk
  for unknown reason the CROSSENV variable is empty, use $(CROSSENV_PATH)/bin/activate instead

* ffsync: fix build of cross/cryptography

- remove wrong definitions of OPENSSL_LIB_DIR and OPENSSL_INCLUDE_DIR; correct values are imported from spksrc.crossenv.mk

* spksrc.python-wheel.mk: Fix wheel building

* DTLSSocket: Update from 0.1.16 to 0.2.2

* homeassistant: Fix building bcrupt and cryptography

* spksrc.crossenv.mk: Ensure Python=* has hostpython to avoid errors

* crossenv/requirements-numpy-1.24.4.txt: Definition for numpy

* python.mk: Remove zlib at source to avoir re-building

* dtlssocket: Add -std=c99 for gcc < 5

* python.mk: Erase only $(ARCH)-$(TCVERSION) related files

* deluge: Update wheels and fix build

* boost_1.82: Allow builds from python.mk

* cross/cryptography: crossenv already has cffi and pycparser

* crossenv.mk: Add debug traces to find issue on gh

* crossenv.mk: Testing alternative method to get python version dir

* crossenv.mk: Send msg to status file

* wheel.mk: Remove trailing # comment using $${wheel%% *}

* crossenv.mk: Use $(WORK_DIR)/pip caching dir for crossenv creation

* python-wheels: include wheels in package

* update github build action

- add evaluation of python313 dependent packages
- evaluate packages to build and changed dependency folders in build.yml
  to avoid overflow of bash variables (limited to 1024 chars) (fixes #6284)

* adjust variable GH_DEPENDENCY_FOLDERS

* real fix for #6284

- python dependent packages with name sorted after python31* where removed from packages to build
  (in this context: python-wheels and rdiff-backup)

* fix github prepare script

- add space to separate packages

* crossenv.mk: Miscelanious cleanups

* mariadb-connector-c: Fix compilation for some archs

* fix package name substitution in prepare.sh

* Update mk/spksrc.crossenv.mk - typo

Co-authored-by: hgy59 <[email protected]>

* python311-wheels: Rename to reference exact matching python version

* native/python313: Sync wheels with default crossenv versions

* python312: Initial package

* mariadb-connector-c: Fix build using spksrc.python.mk

* prepare.sh: keep python311-wheels to build

* fix python312/PLIST

* cleanup python

- python311-wheels for ARMv7L (crossenv for numpy-1.22.4) - didn't fix it
- add PLIST files for python312 and python313
- remove lxml==4.9.3 in python311-wheels/src/requirements-crossenv.txt (build only lxml==5.3.0)

* fix INFO file generation

- use explicit /bin/sh to fix language specific description generation

* python packages: further cleanup

- update unsupported archs in cross Makefiles
- python311-wheels: avoid multiple versions of greenlet

* wheel-env.mk: Bump default abi3 to match py311 with cp37

* crossenv.mk: Reduce total amount of makefile variable definitions

* crossenv.mk: Empty MAKEFLAGS= variable when calling crossenv recipe

* crossenv.mk: Fix missing variable definition

* tc.mk: Enhance meson toolchain file declarations

* wheel.mk: Add meson wheel compiling ability

* python311-wheels: Add new compatible numpy versions

* python312-wheels: Wheel testing package against py312

* Update spk/python312-wheels/Makefile

Co-authored-by: hgy59 <[email protected]>

* numpy-1.25.1: Adding crossenv requirement file

* crossenv.mk: Fix missing definitions for HOST and BUILD_ARCH

* wheel.mk: sort wheelhouse requirement files to avoid duplicates

* python31*-wheels: various adjustments

* restore python311-wheels/src/requirements-crossenv-greenlet-v3.txt

* build.sh: add python312 and python313 to packages to keep

* python.mk: Add xz (lzma) to the library exception list

* python311-wheels: Sync greenlet requirement files with py312-wheels

* python.mk: Use *_EXCLUDE variables to simplify later additions

* python310: Update from version 3.10.15 to 3.10.16

* python311: Update from version 3.11.10 to 3.11.11

* python312-python313: Sync requirement files with py310-311

* python312: Update from version 3.12.7 to 3.12.8

* python313: Update from version 3.13.0 to 3.13.1

* py311-312-wheels: Fix greenlet 3.x support with gcc <= 8

* crossenv.mk: Move PATH and LD_LIBRARY_PATH to be crossenv specific

* define build-python when not in path

* minimize specific crossenv for frozenlist and yarl

* remove unsupported configure args

- --with-system-ffi is not supported (python312, python313)
- --with-system-ffi is "is ignored on this platform" (python311)

* revert fullpath for build-python for python310 and python311

- python310 and python311 require the path to build python for "generate-posix-vars"

* python312-wheels: fix build

- add wheelhouse to package
- many wheels need -std=c11 for gcc4
- llfuse wheel fails to build (temp. excluded)
- remove ARMv5 specific handling
- udpate rpds_py==0.20.0 and remove requirements-crossenv-rpds-py.txt (not referenced)
- add specific crossenv for pydantic_core

* revert unintended change

* spksrc.wheel.mk: add PATH for build of pure-python-wheels

* Revert move of crossenv files temporarely

Created a patch from that reverted comit to first focus
on other issues found then return on this.

* crossenv.mk: Fix PATH and LD_LIBRARY_PATH for native and crossenv

* py310-313: Fix --with-build-python using $(PYTHON_NATIVE)

* py310: re-revert fix --with-build-python only for py310

* crossenv.mk: Move crossenv definitions to spk/python3*

- Simplified shell calls to avoid defining SHELL = /bin/bash
- Added at ifneq ($(wildcard file-wheel file-default)) as at early makefile variable asignation path will not have been determined just yet, thus blocking when trying to determine default version values extracted from variable within requirement files
- Fixed CROSSENV_CONFIG_PATH so it always refer back to $(PYTHON_WORK_DIR) to ensure it works both from spk/python3* and from python related spk projects called using spksrc.python.mk

Co-Author: hgy59 <[email protected]>

* python311-wheels: Update numpy requirements

Numpy fails to build starting with version >= 1.26.
May require to be migrated to use cross/numpy with proper
wheel building including meson toolchain file parameter passing.

* python-wheels: Cleanups and comment regarding late WHEELS definitions

* python-wheels: Add pure python wheel serving as example

* spksrc.wheel.mk: Numerous cleanups & reduced shell calls

* python311-wheels: Fix numpy for x64 DSM-7.1

Fails to build numpy 1.25.2 but ok with 1.25.1

* python311: Add crossenv definition for numpy-1.25.1 for x64 DSM-7.1

* python311-wheels: Extend Pillows dependencies webp, tiff, imagequant

* wheel.mk: Remove unused PATH and LD_LIBRARY_PATH to cross building

* deluge: Update to latest wheel versions

* wheel.mk: pip build environment variable fixes + cleanups

* boost*: Fix loading crossenv python-cc.mk

* wheel.mk: Additional fixes mainly around PATH handling

* tc.mk: Remove sysroot from meson definition as breaking glib

---------

Co-authored-by: hgy59 <[email protected]>

.github/actions/build.sh

@@ -61,7 +61,7 @@ if [ -n "$API_KEY" ] && [ "$PUBLISH" == "true" ]; then
 fi
 
 # Build
-PACKAGES_TO_KEEP="synocli-videodriver ffmpeg5 ffmpeg7 python310 python311"
+PACKAGES_TO_KEEP="synocli-videodriver ffmpeg5 ffmpeg7 python310 python311 python312  python313"
 for package in ${build_packages}
 do
     echo "::group:: ---- build ${package}"

.github/actions/prepare.sh

@@ -5,9 +5,10 @@
 # Evaluate packages to build and referenced source files to download.
 #
 # Functions:
-# - Evaluate all packages to build depending on files defined in ${GH_FILES}.
+# - Build all packages defined by ${USER_SPK_TO_BUILD} and ${GH_SPK_PACKAGES}
+# - Evaluate additional packages to build depending on changed folders defined in ${GH_DEPENDENT_PACKAGES}.
 # - synocli-videodriver is moved to head of packages to build first if triggered by its ffmpeg5-7
-# - python310-311 and ffmpeg5-7 are moved to head of remaining packages to build when triggered by its own or a dependent.
+# - python310-313 and ffmpeg5-7 are moved to head of remaining packages to build when triggered by its own or a dependent.
 # - Referenced native and cross packages of the packages to build are added to the download list.
 
 set -o pipefail
@@ -18,12 +19,8 @@ echo "::group:: ---- find dependent packages"
 make setup-synocommunity
 DEFAULT_TC=$(grep DEFAULT_TC local.mk | cut -f2 -d= | xargs)
 
-# filter for changes made in the spk directories and take unique package name (without spk folder)
-SPK_TO_BUILD+=" "
-SPK_TO_BUILD+=$(echo "${GH_FILES}" | tr ' ' '\n' | grep -oP "^spk/\K[^\/]*" | sort -u | tr '\n' ' ')
-
-# filter for changes made in the cross and native directories and take unique package name (including cross or native folder)
-DEPENDENT_PACKAGES=$(echo "${GH_FILES}" | tr ' ' '\n' | grep -oP "(cross|native)/[^\/]*" | sort -u | tr '\n' ' ')
+# all packages to build from changes or manual definition
+SPK_TO_BUILD="${USER_SPK_TO_BUILD} ${GH_SPK_PACKAGES} "
 
 # get dependency list
 # dependencies in this list include the cross or native folder (i.e. native/python cross/glib)
@@ -37,20 +34,20 @@ do
 done
 
 # search for dependent spk packages
-for package in ${DEPENDENT_PACKAGES}
+for package in ${GH_DEPENDENCY_FOLDERS}
 do
     echo "===> Searching for dependent package: ${package}"
     packages=$(echo "${DEPENDENCY_LIST}" | grep " ${package} " | grep -o ".*:" | tr ':' ' ' | sort -u | tr '\n' ' ')
     echo "===> Found: ${packages}"
-    SPK_TO_BUILD+=${packages}
+    SPK_TO_BUILD+=" ${packages}"
 done
 
 # fix for packages with different names
-if [ "$(echo ${SPK_TO_BUILD} | grep -ow nzbdrone)" != "" ]; then
-    SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -vw "nzbdrone" | tr '\n' ' ')" sonarr3"
+if [ "$(echo ${SPK_TO_BUILD} | grep -o ' nzbdrone ')" != "" ]; then
+    SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -v "^nzbdrone$" | tr '\n' ' ')" sonarr3"
 fi
-if [ "$(echo ${SPK_TO_BUILD} | grep -ow python)" != "" ]; then
-    SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -vw "python" | tr '\n' ' ')" python2"
+if [ "$(echo ${SPK_TO_BUILD} | grep -o ' python ')" != "" ]; then
+    SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -v "^python$" | tr '\n' ' ')" python2"
 fi
 
 # remove duplicate packages
@@ -65,7 +62,7 @@ for i in {5..7}; do
     for package in ${packages}
     do
         if [ "$(echo ffmpeg${i} ${ffmpeg_dependent_packages} | grep -ow ${package})" != "" ]; then
-            packages_without_ffmpeg=$(echo "${packages}" | tr ' ' '\n' | grep -v "ffmpeg${i}" | tr '\n' ' ')
+            packages_without_ffmpeg=$(echo "${packages}" | tr ' ' '\n' | grep -v "^ffmpeg${i}\$" | tr '\n' ' ')
             packages="ffmpeg${i} ${packages_without_ffmpeg}"
             break
         fi
@@ -80,22 +77,22 @@ videodrv_dependent_packages=$(find spk/ -maxdepth 2 -mindepth 2 -name "Makefile"
 for package in ${packages}
 do
     if [ "$(echo synocli-videodriver ${videodrv_dependent_packages} | grep -ow ${package})" != "" ]; then
-        packages_without_videodrv=$(echo "${packages}" | tr ' ' '\n' | grep -v "synocli-videodriver" | tr '\n' ' ')
+        packages_without_videodrv=$(echo "${packages}" | tr ' ' '\n' | grep -v "^synocli-videodriver\$" | tr '\n' ' ')
         packages="synocli-videodriver ${packages_without_videodrv}"
         break
     fi
 done
 
-# for python (310, 311) find all packages that depend on them
-for py in python310 python311; do
+# for python (310, 311, 312, 313) find all packages that depend on them
+for py in python310 python311 python312 python313; do
     python_dependent_packages=$(find spk/ -maxdepth 2 -mindepth 2 -name "Makefile" -exec grep -Ho "PYTHON_PACKAGE = ${py}" {} \; | grep -Po ".*spk/\K[^/]*" | sort | tr '\n' ' ')
 
     # If packages contain a package that depends on python (or is python), then ensure
     # relevant python spk is first in list
     for package in ${packages}
     do
         if [ "$(echo ${py} ${python_dependent_packages} | grep -ow ${package})" != "" ]; then
-            packages_without_python=$(echo "${packages}" | tr ' ' '\n' | grep -v "${py}" | tr '\n' ' ')
+            packages_without_python=$(echo "${packages}" | tr ' ' '\n' | grep -v "^${py}\$" | tr '\n' ' ')
             packages="${py} ${packages_without_python}"
             break
         fi

.github/workflows/build.yml

@@ -84,26 +84,31 @@ jobs:
           fetch-depth: 0
           persist-credentials: false
 
-      - name: Get changed files for pull request
+      - name: Get changed spk_packages and dependency_folders for pull request
         if: github.event_name == 'pull_request'
-        id: getfile_pr
+        id: getchanges_pr
         run: |
-          git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | xargs
-          echo "files=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | xargs)" >> $GITHUB_OUTPUT
+          git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs
+          echo "spk_packages=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
+          git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs
+          echo "dependency_folders=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
 
-      - name: Get changed files for push
+      - name: Get changed spk_packages and dependency_folders of last commit for push
         if: github.event_name == 'push'
-        id: getfile
+        id: getchanges_push
         run: |
-          git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | xargs
-          echo "files=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | xargs)" >> $GITHUB_OUTPUT
+          git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs
+          echo "spk_packages=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
+          git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs
+          echo "dependency_folders=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
 
       - name: Evaluate dependencies
         id: dependencies
         run: ./.github/actions/prepare.sh
         env:
-          GH_FILES: ${{ steps.getfile.outputs.files }} ${{ steps.getfile_pr.outputs.files }}
-          SPK_TO_BUILD: ${{ github.event.inputs.package }}
+          GH_SPK_PACKAGES: ${{ steps.getchanges_push.outputs.spk_packages }} ${{ steps.getchanges_pr.outputs.spk_packages }}
+          GH_DEPENDENCY_FOLDERS: ${{ steps.getchanges_push.outputs.dependency_folders }} ${{ steps.getchanges_pr.outputs.dependency_folders }}
+          USER_SPK_TO_BUILD: ${{ github.event.inputs.package }}
 
       - name: Cache downloaded files
         uses: actions/cache@v4

cross/boost/Makefile

@@ -24,7 +24,7 @@ WITH_PYTHON_LIBRARY =
 ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
 WITH_PYTHON_LIBRARY = 1
 # The packages depending on boost with python, must build python before.
-include $(WORK_DIR)/python-cc.mk
+include $(or $(wildcard $(WORK_DIR)/crossenv/build/python-cc.mk), $(wildcard $(PYTHON_PACKAGE_WORK_DIR)/crossenv/build/python-cc.mk))
 PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
 else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
 # exclude python library if "all" selected

cross/boost_1.68/Makefile

@@ -26,7 +26,7 @@ WITH_PYTHON_LIBRARY =
 ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
 WITH_PYTHON_LIBRARY = 1
 # The packages depending on boost with python, must build python before.
-include $(WORK_DIR)/python-cc.mk
+include $(or $(wildcard $(WORK_DIR)/crossenv/build/python-cc.mk), $(wildcard $(PYTHON_PACKAGE_WORK_DIR)/crossenv/build/python-cc.mk))
 PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
 else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
 # exclude python library if "all" selected

cross/boost_1.70/Makefile

@@ -27,7 +27,7 @@ WITH_PYTHON_LIBRARY =
 ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
 WITH_PYTHON_LIBRARY = 1
 # The packages depending on boost with python, must build python before.
-include $(WORK_DIR)/python-cc.mk
+include $(or $(wildcard $(WORK_DIR)/crossenv/build/python-cc.mk), $(wildcard $(PYTHON_PACKAGE_WORK_DIR)/crossenv/build/python-cc.mk))
 PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
 else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
 # exclude python library if "all" selected

cross/boost_1.78/Makefile

@@ -24,7 +24,7 @@ WITH_PYTHON_LIBRARY =
 ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
 WITH_PYTHON_LIBRARY = 1
 # The packages depending on boost with python, must build python before.
-include $(WORK_DIR)/python-cc.mk
+include $(or $(wildcard $(WORK_DIR)/crossenv/build/python-cc.mk), $(wildcard $(PYTHON_PACKAGE_WORK_DIR)/crossenv/build/python-cc.mk))
 PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
 else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
 # exclude python library if "all" selected

cross/boost_1.82/Makefile

@@ -24,7 +24,7 @@ WITH_PYTHON_LIBRARY =
 ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
 WITH_PYTHON_LIBRARY = 1
 # The packages depending on boost with python, must build python before.
-include $(WORK_DIR)/python-cc.mk
+include $(or $(wildcard $(WORK_DIR)/crossenv/build/python-cc.mk), $(wildcard $(PYTHON_PACKAGE_WORK_DIR)/crossenv/build/python-cc.mk))
 PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
 else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
 # exclude python library if "all" selected
@@ -43,7 +43,7 @@ ADDITIONAL_CXXFLAGS = -Wno-deprecated-declarations
 
 ifneq ($(strip $(WITH_PYTHON_LIBRARY)),)
 CONFIGURE_ARGS += --with-python=$(WORK_DIR)/../../../native/$(PYTHON_NAME)/work-native/install/usr/local/bin/$(PYTHON_NAME)
-ADDITIONAL_CXXFLAGS += -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) -fpermissive
+ADDITIONAL_CXXFLAGS += -I$(or $(PYTHON_STAGING_INSTALL_PREFIX),$(STAGING_INSTALL_PREFIX))/$(PYTHON_INC_DIR) -fpermissive
 endif
 
 include ../../mk/spksrc.common.mk
@@ -75,7 +75,7 @@ boost_pre_compile:
 	@rm -rf $(WORK_DIR)/$(PKG_DIR)/user-config.jam
 	@echo "using gcc : ${TC_GCC} : ${CXX} : <address-model>\"$(ADDRESS_MODEL)\" <cflags>\"$(CFLAGS)\" <cxxflags>\"$(CXXFLAGS) $(ADDITIONAL_CXXFLAGS)\" <linkflags>\"$(LDFLAGS)\" <link>\"shared\" ; " > $(WORK_DIR)/$(PKG_DIR)/user-config.jam
 ifneq ($(strip $(WITH_PYTHON_LIBRARY)),)
-	@echo "using python : $(PYTHON_VERSION) : : $(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam
+	@echo "using python : $(PYTHON_VERSION) : : $(or $(PYTHON_STAGING_INSTALL_PREFIX),$(STAGING_INSTALL_PREFIX))/$(PYTHON_INC_DIR) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam
 endif
 	@echo "project : requirements <cxxflags>$(OFLAGS) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam
 

cross/cryptography/Makefile

@@ -2,20 +2,13 @@ PKG_NAME = cryptography
 PKG_VERS = 41.0.3
 PKG_EXT = tar.gz
 PKG_DIST_NAME = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
-PKG_DIST_SITE = https://files.pythonhosted.org/packages/source/c/$(PKG_NAME)
+PKG_DIST_SITE = https://files.pythonhosted.org/packages/source/c/cryptography
 PKG_DIR = $(PKG_NAME)-$(PKG_VERS)
 
-DEPENDS = cross/openssl3 cross/cffi
+DEPENDS = cross/openssl3 cross/libffi
 
 HOMEPAGE = https://github.com/pyca/cryptography
 COMMENT  = Provide cryptographic recipes and primitives to Python developers
 LICENSE  = BSD/ASL2
 
-###
-### This cross/cryptography is needed to generate python-limited abi3 wheels
-### Ref: https://github.com/pyca/cryptography/issues/7671
-###
-
-WHEELS_BUILD_ARGS = --py-limited-api=$(PYTHON_LIMITED_API)
-
 include ../../mk/spksrc.python-wheel.mk

cross/dtlssocket/Makefile

@@ -1,18 +1,22 @@
-PKG_NAME = DTLSSocket
-PKG_VERS = 0.1.16
+PKG_NAME = dtlssocket
+PKG_VERS = 0.2.2
 PKG_EXT = tar.gz
 PKG_DIST_NAME = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
-PKG_DIST_SITE = https://files.pythonhosted.org/packages/58/42/0a0442118096eb9fbc9dc70b45aee2957f7546b80545e2a05bd839380519
+PKG_DIST_SITE = https://files.pythonhosted.org/packages/97/b6/702c4197c37c7036d0b7d67a2963c83a4b7098525959566dc6e2284c9f89
 PKG_DIR = $(PKG_NAME)-$(PKG_VERS)
 
-BUILD_DEPENDS = cross/cython
-
 HOMEPAGE = https://git.fslab.de/jkonra2m/tinydtls-cython
 COMMENT  = DTLSSocket is a Cython wrapper for tinydtls with a Socket like interface.
 LICENSE  = Eclipse Public License 1.0
 
 PRE_CONFIGURE_TARGET = dtlssocket_pre_configure
 
+include ../../mk/spksrc.common.mk
+
+ifeq ($(call version_le, $(TC_GCC), 5),1)
+ADDITIONAL_CFLAGS = -std=c99
+endif
+
 include ../../mk/spksrc.python-wheel.mk
 
 .PHONY: dtlssocket_pre_configure

cross/dtlssocket/digests

@@ -1,3 +1,3 @@
-DTLSSocket-0.1.16.tar.gz SHA1 9b4eaa3e9a949f202844a27ba54b2f05d73aa410
-DTLSSocket-0.1.16.tar.gz SHA256 30b108ae45fce1c033e3ef6c2ddd7e7418062ae374228e3a7fa969b254366a39
-DTLSSocket-0.1.16.tar.gz MD5 15dfeca9678e381c7bb6e11e4c864fd5
+dtlssocket-0.2.2.tar.gz SHA1 0be2a954ca56fe7cca3a9fbc5a9e65194f0ab5a5
+dtlssocket-0.2.2.tar.gz SHA256 4e76d715726e0c46dc09e35da9b6717b063c2389b06c1723decc3ba38b7387f4
+dtlssocket-0.2.2.tar.gz MD5 d1f686febd12e67ec7a2efe81bf1eaee

cross/mariadb-connector-c/Makefile

@@ -1,5 +1,7 @@
 PKG_NAME = mariadb-connector-c
-PKG_VERS = 3.3.5
+# Version 3.3.11 is the last known version to work with aarch64-6.2.4
+# Assembly error: Error: CFI instruction used without previous .cfi_startproc
+PKG_VERS = 3.3.11
 PKG_EXT = tar.gz
 PKG_DIST_NAME = v$(PKG_VERS).$(PKG_EXT)
 PKG_DIST_FILE = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
@@ -10,23 +12,47 @@ HOMEPAGE = https://mariadb.com/kb/en/mariadb-connector-c/
 COMMENT  = MariaDB C Connector
 LICENSE  = LGPL
 
-#USE_NATIVE_CMAKE_LEGACY = 1
-#CMAKE_USE_TOOLCHAIN_FILE = OFF
 CMAKE_DISABLE_EXE_LINKER_FLAGS = 1
 
 include ../../mk/spksrc.cross-cmake.mk
 
-ifeq ($(call version_lt, $(TC_GCC), 5.0),1)
+ifeq ($(call version_gt, $(TC_GCC), 4.9),1)
+ADDITIONAL_CFLAGS += -Wno-error=maybe-uninitialized
+# fails on evansport-6.2.4
+ADDITIONAL_CFLAGS += -Wno-error=strict-aliasing
+endif
+
+ifeq ($(call version_lt, $(TC_GCC), 4.9),1)
 ADDITIONAL_CFLAGS += -std=gnu99
+# fails on 88f6281-6.2.4
+ADDITIONAL_CFLAGS += -Wno-error=missing-field-initializers
+ADDITIONAL_CFLAGS += -Wno-error=uninitialized
+# fails on hi3535-6.2.4
+ADDITIONAL_CFLAGS += -Wno-error=array-bounds
 endif
 
 # MySQL specific:
 CMAKE_ARGS += -DWITH_UNIT_TESTS=OFF
 CMAKE_ARGS += -DDEFAULT_CHARSET=utf8
 CMAKE_ARGS += -DINSTALL_LIBDIR=lib
 
+# Creates the following two synlinks:
+#   - libmysqlclient.so -> libmariadb.so.3
+#   - libmysqlclient_r.so -> libmariadb.so.3
+# Which in turns conflict with mysql-connector-c
+#CMAKE_ARGS += -DWITH_MYSQLCOMPAT=ON
+
+# Does not handle pkgconfig well and fails to
+# find openssl3 include and libraries using
+# spksrc.python.mk thus using OPENSSL_STAGING_PREFIX
 DEPENDS += cross/openssl3
 CMAKE_ARGS += -DWITH_SSL=ON
+ifneq ($(wildcard $(OPENSSL_STAGING_PREFIX)),)
+CMAKE_ARGS += -DOPENSSL_CRYPTO_LIBRARY:FILEPATH=$(OPENSSL_STAGING_PREFIX)/lib/libcrypto.so
+CMAKE_ARGS += -DOPENSSL_SSL_LIBRARY:FILEPATH=$(OPENSSL_STAGING_PREFIX)/lib/libssl.so
+CMAKE_ARGS += -DOPENSSL_INCLUDE_DIR:PATH=$(OPENSSL_STAGING_PREFIX)/include
+endif
+
 DEPENDS += cross/zlib
 CMAKE_ARGS += -DWITH_EXTERNAL_ZLIB=ON
 DEPENDS += cross/libiconv

cross/mariadb-connector-c/digests

@@ -1,3 +1,3 @@
-mariadb-connector-c-3.3.5.tar.gz SHA1 29ad65e81a87e41a93e8984f2c3d1662dcec7c1c
-mariadb-connector-c-3.3.5.tar.gz SHA256 c0fda1fa6e52dc85de27156cd847088a72d40d9de6514f7efa57c8d93134a54c
-mariadb-connector-c-3.3.5.tar.gz MD5 b8967cf63d4fc5660f230762c0e39ee2
+mariadb-connector-c-3.3.11.tar.gz SHA1 2fa1fe042bcc65297d6d22196685701faf3e96d1
+mariadb-connector-c-3.3.11.tar.gz SHA256 6ecada1f3fc32ad95efcd2c982b319191f5e6479f3f521bc4b714e6158f5e415
+mariadb-connector-c-3.3.11.tar.gz MD5 700afe8ebff69bdf66b3c37747d1d527