Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build library and also Python wheels in CI #390

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
bf8cf9a
add Python build detritus to `.gitignore`
zacharyburnett Nov 11, 2024
59b63a5
add SWIG to `build-system.requires`
zacharyburnett Nov 11, 2024
e166179
set `BUILD_TESTS=off` when building wheels
zacharyburnett Nov 11, 2024
feee1a3
add workflow to automatically build wheels and upload as artifacts
zacharyburnett Nov 11, 2024
9614d07
also test library building with CMake
zacharyburnett Nov 11, 2024
7649588
add workflow to test pip-installed package
zacharyburnett Nov 11, 2024
f173b27
add build/ to git ignore
zacharyburnett Nov 11, 2024
306cb49
remove job specification from build (`--parallel=8` -> `--parallel`)
zacharyburnett Nov 11, 2024
5426608
run ctest
zacharyburnett Nov 11, 2024
18b3139
update googletest version
zacharyburnett Nov 12, 2024
a2c8a64
explicitly set GOOGLETEST_VERSION
zacharyburnett Nov 12, 2024
3cdefa4
for some reason the universal wheels aren't universal
zacharyburnett Nov 12, 2024
ab525c0
set `CTEST_OUTPUT_ON_FAILURE=ON`
zacharyburnett Nov 14, 2024
d3df01d
explicitly include googletest in PEP517 wheel build
zacharyburnett Nov 14, 2024
fdfcd74
default to not building tests
zacharyburnett Nov 14, 2024
d2ce2ac
test without parallel build on Linux; perhaps the Linux runner is ove…
zacharyburnett Nov 14, 2024
cb65e68
remove googletest from wheel build; it doesn't seem to make a difference
zacharyburnett Nov 14, 2024
a26cf9d
also run tests on macOS Intel runners
zacharyburnett Nov 14, 2024
a0924ea
exclude macOS Python <= 3.12 from pytest and wheel building
zacharyburnett Nov 15, 2024
bd5f6b3
explain why --parallel test building is disabled on Linux
zacharyburnett Nov 15, 2024
8330fdc
add comments to gitignore
zacharyburnett Nov 15, 2024
9108afb
fill out more Python versions for testing
zacharyburnett Nov 15, 2024
aacb92e
upload to PyPI (only on release)
zacharyburnett Nov 20, 2024
726306f
add wheel delocation step
zacharyburnett Dec 2, 2024
3cef06a
use `repairwheel` to delocate wheel on both platforms
zacharyburnett Dec 3, 2024
0446b38
set timeout to 30 minutes
zacharyburnett Dec 5, 2024
74a0e88
add concurrency lock
zacharyburnett Dec 5, 2024
f97d130
read-all permissions
zacharyburnett Dec 5, 2024
6322abf
use `cibuildwheel` with AlmaLinux9 `manylinux`
zacharyburnett Dec 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: build
zacharyburnett marked this conversation as resolved.
Show resolved Hide resolved

on:
pull_request:
push:
branches:
- master
release:
types:
- released
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true

permissions: read-all

jobs:
zacharyburnett marked this conversation as resolved.
Show resolved Hide resolved
zacharyburnett marked this conversation as resolved.
Show resolved Hide resolved
cmake:
strategy:
matrix:
runs-on:
- ubuntu-latest
# macos-13 is an intel runner, macos-14 is apple silicon
- macos-13
- macos-latest
fail-fast: false
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 30
steps:
- uses: zacharyburnett/setup-abseil-cpp@de39f445295c887839e30c864ffbbb1c0231bc83 # 1.0.5
with:
cmake-build-args: "-DCMAKE_CXX_STANDARD=17 -DABSL_PROPAGATE_CXX_STD=ON -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off -DCMAKE_POSITION_INDEPENDENT_CODE=ON"
abseil-version: "20240722.0"
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- run: mkdir build
- run: cmake -DCMAKE_CXX_STANDARD=17 -DCMAKE_PREFIX_PATH=/usr/local/ -DBUILD_TESTS=OFF ..
working-directory: build/
- run: sudo cmake --build . --parallel --target=install
working-directory: build/
python-wheels:
strategy:
matrix:
runs-on:
- ubuntu-latest
# macos-13 is an intel runner, macos-14 is apple silicon
- macos-13
- macos-latest
fail-fast: false
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 30
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: pypa/cibuildwheel@ee63bf16da6cddfb925f542f2c7b59ad50e93969 # v2.22.0
with:
package-dir: .
output-dir: dist/
config-file: "{package}/pyproject.toml"
- uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: s2geometry-${{ runner.os }}-${{ runner.arch }}
path: ./dist/*.whl
zacharyburnett marked this conversation as resolved.
Show resolved Hide resolved
- if: (github.event_name == 'release') && (github.event.action == 'released')
uses: pypa/gh-action-pypi-publish@15c56dba361d8335944d31a2ecd17d700fc7bcbc # v1.12.2
106 changes: 106 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
name: test

on:
pull_request:
push:
branches:
- master
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true

permissions: read-all

env:
GOOGLETEST_VERSION: "1.15.2"

jobs:
ctest:
strategy:
matrix:
runs-on:
- ubuntu-latest
# macos-13 is an intel runner, macos-14 is apple silicon
- macos-13
- macos-latest
fail-fast: false
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 30
env:
CTEST_OUTPUT_ON_FAILURE: ON
steps:
- uses: zacharyburnett/setup-abseil-cpp@de39f445295c887839e30c864ffbbb1c0231bc83 # 1.0.5
with:
cmake-build-args: "-DCMAKE_CXX_STANDARD=17 -DABSL_PROPAGATE_CXX_STD=ON -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off -DCMAKE_POSITION_INDEPENDENT_CODE=ON"
abseil-version: "20240722.0"
- name: retrieve googletest v${{ env.GOOGLETEST_VERSION }}
run: |
wget https://github.com/google/googletest/releases/download/v${{ env.GOOGLETEST_VERSION }}/googletest-${{ env.GOOGLETEST_VERSION }}.tar.gz
tar -xzvf googletest-${{ env.GOOGLETEST_VERSION }}.tar.gz
echo GOOGLETEST_ROOT=${{ runner.temp }}/googletest-${{ env.GOOGLETEST_VERSION }} >> $GITHUB_ENV
working-directory: ${{ runner.temp }}
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- run: mkdir build
- run: cmake -DCMAKE_CXX_STANDARD=17 -DCMAKE_PREFIX_PATH=/usr/local/ -DGOOGLETEST_ROOT=${{ env.GOOGLETEST_ROOT }} -DGOOGLETEST_VERSION=${{ env.GOOGLETEST_VERSION }} ..
working-directory: build/
- run: cmake --build . ${{ runner.os == 'macOS' && '--parallel' || '' }}
# building tests with `--parallel` is disabled on Linux because GitHub seems to automatically cancel the job due to resource limits(?)
working-directory: build/
- if: always()
run: cmake --build . --parallel --target=test
jmr marked this conversation as resolved.
Show resolved Hide resolved
working-directory: build/
pytest:
strategy:
matrix:
runs-on:
- ubuntu-latest
# macos-13 is an intel runner, macos-14 is apple silicon
- macos-13
- macos-latest
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
exclude:
# importing the package from the wheel instantly segfaults on macOS Python <= 3.12
- runs-on: macos-latest
python-version: "3.8"
- runs-on: macos-latest
python-version: "3.9"
- runs-on: macos-latest
python-version: "3.10"
- runs-on: macos-latest
python-version: "3.11"
- runs-on: macos-latest
python-version: "3.12"
- runs-on: macos-13
python-version: "3.8"
- runs-on: macos-13
python-version: "3.9"
- runs-on: macos-13
python-version: "3.10"
- runs-on: macos-13
python-version: "3.11"
- runs-on: macos-13
python-version: "3.12"
fail-fast: false
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 30
steps:
- uses: zacharyburnett/setup-abseil-cpp@de39f445295c887839e30c864ffbbb1c0231bc83 # 1.0.5
with:
cmake-build-args: "-DCMAKE_CXX_STANDARD=17 -DABSL_PROPAGATE_CXX_STD=ON -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off -DCMAKE_POSITION_INDEPENDENT_CODE=ON"
abseil-version: "20240722.0"
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: ${{ matrix.python-version }}
- run: pip install .
- run: pip install pytest
- run: pytest

7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
build/

# Python wheels
dist/
jmr marked this conversation as resolved.
Show resolved Hide resolved

# Python package metadata
*.egg-info
jmr marked this conversation as resolved.
Show resolved Hide resolved
18 changes: 18 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ requires = [
"setuptools",
"setuptools_scm[toml]",
"cmake_build_extension",
"swig",
]
build-backend = "setuptools.build_meta"

Expand All @@ -56,3 +57,20 @@ namespaces = false
[tool.setuptools.package-dir]
"" = "src"

[tool.cibuildwheel]
manylinux-x86_64-image = "quay.io/pypa/manylinux_2_34_x86_64"
test-requires = "pytest"
test-command = "pytest {project}/tests"

[tool.cibuildwheel.environment]
MACOSX_DEPLOYMENT_TARGET = 14.0

[tool.cibuildwheel.linux]
#repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"
# the EPEL9 package `abseil-cpp` is out of date, so we have to build `abseil-cpp` from source
before-all = "yum install -y gflags-devel glog-devel gtest-devel openssl-devel && git clone https://github.com/abseil/abseil-cpp.git && cd abseil-cpp && mkdir build && cd build && cmake -DCMAKE_CXX_STANDARD=17 -DABSL_PROPAGATE_CXX_STD=ON -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off .. && cmake --build . --parallel --target=install"

[tool.cibuildwheel.macos]
#repair-wheel-command = "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"
before-all = "brew install gflags glog openssl abseil"

20 changes: 12 additions & 8 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
import os
import sys
from pathlib import Path

import cmake_build_extension
import setuptools


SETUPTOOLS_CMAKE_OPTIONS = os.environ.get("SETUPTOOLS_CMAKE_OPTIONS", "-DBUILD_TESTS=OFF").split()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's using this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using this to test whether inserting GOOGLETEST_ROOT into the cmake options would solve the segfault; I can remove it if desired

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, just revert this file unless you really need the changes.


setuptools.setup(
ext_modules=[
cmake_build_extension.CMakeExtension(
Expand All @@ -16,14 +19,15 @@
# (it could be a subfolder)
source_dir=str(Path(__file__).parent.absolute()),
cmake_configure_options=[
# This option points CMake to the right Python interpreter, and helps
# the logic of FindPython3.cmake to find the active version
f"-DPython3_ROOT_DIR={Path(sys.prefix)}",
'-DCALL_FROM_SETUP_PY:BOOL=ON',
'-DBUILD_SHARED_LIBS:BOOL=OFF',
'-DCMAKE_POSITION_INDEPENDENT_CODE=ON',
'-DWITH_PYTHON=ON'
]
# This option points CMake to the right Python interpreter, and helps
# the logic of FindPython3.cmake to find the active version
f"-DPython3_ROOT_DIR={Path(sys.prefix)}",
"-DCALL_FROM_SETUP_PY:BOOL=ON",
"-DBUILD_SHARED_LIBS:BOOL=OFF",
"-DCMAKE_POSITION_INDEPENDENT_CODE=ON",
"-DWITH_PYTHON=ON",
*SETUPTOOLS_CMAKE_OPTIONS,
],
)
],
cmdclass=dict(
Expand Down
Loading