Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/2.7.6 #45

Merged
merged 57 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
09bd849
WIP:Security fix to limit which folders/files can be accessed
jmeasure Jun 20, 2024
ffb23fe
WIP: Fixing bug in CMakeList for boost
jmeasure Jun 20, 2024
419fb6e
WIP: Boost is annoying, wrote my own functions, anyone can feel free …
jmeasure Jun 20, 2024
04a36b2
Merge branch 'main' of https://github.com/ukaea/UDA into feature/getb…
jmeasure Jun 20, 2024
5165e57
missing newline on split string declaration
jmeasure Jun 20, 2024
e3df5b2
WIP: fixing relating to strict nature of comipilation
jmeasure Jun 20, 2024
eae1421
umm maybe use instead of /home/jmeasure...
jmeasure Jun 20, 2024
d9ca3b9
Adding check for more .. in folder path than folder names
jmeasure Jun 20, 2024
fbe5650
bug RAISE_PLUGIN_ERROR returns a int so incompatible with std::string…
jmeasure Jun 20, 2024
aeba019
Adam adding filesystem requirements to CMakelist, bytes plugin update…
jmeasure Jun 21, 2024
d5a37ba
Adding Boost, updating mast.l.cfg to populate the allowed paths
jmeasure Jun 21, 2024
a5165bc
Update cmake.yml to fix JM's mistakes
adam-parker1 Jun 21, 2024
2b0fd53
Changing check_allowed_path arg to const char*, and allowed paths to …
jmeasure Jun 24, 2024
1c8f1cd
Fixing AP's misatakes
jmeasure Jun 24, 2024
1d2b28e
fixing typo in mast.l.cfg and changing boost::algorithm::split to boo…
jmeasure Jun 27, 2024
8986f0e
Fixing Code breaking bugs, adding a space, removing a blank line, and…
jmeasure Jun 27, 2024
7c4176f
Suprious space entered to blank line, the world can rest easy now it …
jmeasure Jun 27, 2024
a77ad39
Removing boost sytstem dependency as we are only using the header
jmeasure Jun 27, 2024
6a0d95c
Merge pull request #41 from ukaea/feature/getbytes_security
jholloc Jun 27, 2024
43a61f6
Feature/python wheels (#42)
stephen-dixon Jul 1, 2024
b72a696
fixing typo in github workflow
stephen-dixon Jul 1, 2024
0561259
fixing typo in github workflow
stephen-dixon Jul 1, 2024
89519bc
changing testpypi repo from 'ukaea_pyuda_test' to 'uda'
stephen-dixon Jul 1, 2024
199cefd
correcting pypi project url
stephen-dixon Jul 1, 2024
e7a2c47
enabling ssl client authentication in pyuda wheel builds
stephen-dixon Jul 4, 2024
c59769d
updating project links in pyuda wheel
stephen-dixon Jul 5, 2024
579ee54
adding wrapper functions to revert breaking api changes
stephen-dixon Jul 8, 2024
49ff725
test pushing wheels to pypi
stephen-dixon Jul 8, 2024
991b351
test pushing wheels to pypi
stephen-dixon Jul 8, 2024
0b03eae
test pushing wheels to pypi
stephen-dixon Jul 8, 2024
6f75d06
updaing centos7 mirrorlist for manylinux2014 pyuda wheel
stephen-dixon Jul 8, 2024
3e7bb1a
updaing centos7 mirrorlist for manylinux2014 pyuda wheel
stephen-dixon Jul 8, 2024
9cb3bc4
testing pushing pyuda wheels to pypi
stephen-dixon Jul 8, 2024
bdce45c
correcting numpy version range in pyuda pyproject.toml
stephen-dixon Jul 8, 2024
ca9e978
reverting pyuda pypi action to only run on tagged releases
stephen-dixon Jul 8, 2024
8d849c0
adding "uda" alias to pyuda
stephen-dixon Jul 25, 2024
13a4b85
adding "uda" alias to pyuda
stephen-dixon Jul 25, 2024
e2fb5c8
adding "uda" alias to pyuda
stephen-dixon Jul 25, 2024
3c10110
adding git attributes to populate cmake version info in release tarballs
stephen-dixon Aug 6, 2024
f585e8f
adding legacy headers
stephen-dixon Aug 7, 2024
7f13f6e
removing client flags from api functions
stephen-dixon Aug 7, 2024
72938a4
removing client flags from api calls in java wrapper
stephen-dixon Aug 7, 2024
fc7970b
removing client flags from api functions in uda plugin
stephen-dixon Aug 7, 2024
3ad2165
removing client flags from api functions in uda plugin
stephen-dixon Aug 7, 2024
9d0b10d
removing erroneous inline keyword in legacy_client.h declarations
stephen-dixon Aug 7, 2024
7925cd7
renaming internal uses of api functions previously using the client_f…
stephen-dixon Aug 7, 2024
b0bae6c
tidying up legacy headers
stephen-dixon Aug 7, 2024
6053ce1
tidying up legacy headers
stephen-dixon Aug 7, 2024
33a3056
tidying up legacy headers
stephen-dixon Aug 7, 2024
a30d89f
tidying up legacy headers
stephen-dixon Aug 7, 2024
1f5202a
installing pkgconfig files for client only build
stephen-dixon Aug 7, 2024
283ee57
using oldest-supported-numpy for build dependencies in pyuda pyprojec…
stephen-dixon Aug 7, 2024
5a51c62
using oldest-supported-numpy for build dependencies in pyuda pyprojec…
stephen-dixon Aug 7, 2024
88f600d
removing some dead code
stephen-dixon Aug 8, 2024
13596f1
Fixing client segfault when data is returned with non-zero rank but n…
jholloc Aug 13, 2024
f9dfc71
reverting openSSL v3 syntax to v1.1 compatible syntax for udaServerSS…
stephen-dixon Aug 22, 2024
d8378c3
updating c++11 flag to c++17 in pkgconfig files for the cpp wrapper
stephen-dixon Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
CMakeLists.txt export-subst
166 changes: 166 additions & 0 deletions .github/workflows/build_test_wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
name: test building and upload wheels to testpypi

on:
push:
branches:
- release/*
- main

jobs:
build_wheels:
# name: build wheels on ${{matrix.os}} ${{matrix.arch}}
name: build wheels on ${{matrix.build-platform[0]}} ${{matrix.build-platform[1]}}
# runs-on: ${{matrix.os}}
runs-on: ${{matrix.build-platform[0]}}
strategy:
fail-fast: false
matrix:
build-platform:
# skip longer-running builds
# - [ubuntu-latest, x86_64, manylinux2014_x86_64]
- [ubuntu-latest, x86_64, manylinux_2_28_x86_64]
# - [ubuntu-latest,aarch64,manylinux_2_28_aarch64]
- [macos-13, x86_64, macosx_x86_64]
- [macos-14, arm64, macosx_arm64]

steps:
- uses: actions/checkout@v3
# need git tags available for setuptools_scm to grab tags? will lead to slower checkout
with:
fetch-depth: 0

- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
with:
platforms: all

- name: Build manylinux2014 wheels
if: startswith(matrix.build-platform[2], 'manylinux2014')
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_BUILD: cp*-manylinux*
CIBW_SKIP: cp*-musllinux* *-musllinux*
CIBW_BEFORE_ALL: >
yum update -y &&
yum install -y wget openssl-devel libxml2-devel libtirpc-devel &&
cd /tmp &&
wget https://github.com/fmtlib/fmt/archive/refs/tags/10.0.0.tar.gz &&
tar xzf 10.0.0.tar.gz &&
cd fmt-10.0.0 &&
cmake -Bbuild -H. -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build -j --config Release --target install &&
cd /tmp &&
wget https://github.com/gabime/spdlog/archive/refs/tags/v1.11.0.tar.gz &&
tar xzf v1.11.0.tar.gz &&
cd spdlog-1.11.0 &&
cmake -Bbuild -H. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build -j --config Release --target install &&
cd /tmp &&
wget https://github.com/capnproto/capnproto/archive/refs/tags/v0.10.4.tar.gz &&
tar xzf v0.10.4.tar.gz &&
cd capnproto-0.10.4 &&
cmake -Bbuild -H. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build &&
cmake --install build &&
cd /tmp &&
wget https://boostorg.jfrog.io/artifactory/main/release/1.80.0/source/boost_1_80_0.tar.gz &&
tar xzf boost_1_80_0.tar.gz &&
cd boost_1_80_0 &&
./bootstrap.sh --prefix=/usr/local &&
./b2 --without-python --prefix=/usr/local install &&
cd /project &&
cmake -B build -DNO_JAVA_WRAPPER=ON -DBUILD_SHARED_LIBS=ON -DSSLAUTHENTICATION=ON -DENABLE_CAPNP=ON -DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r /usr/local/python_installer/* /project/source/wrappers/python/

- name: Build manylinux_2_28 wheels
if: startswith(matrix.build-platform[2], 'manylinux_2_28')
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_28
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_BUILD: cp*-manylinux*
CIBW_SKIP: cp*-musllinux* *-musllinux*
CIBW_BEFORE_ALL: >
dnf update -y &&
dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm &&
dnf install -y boost-devel openssl-devel libxml2-devel libtirpc-devel
fmt fmt-devel spdlog spdlog-devel capnproto capnproto-devel &&
cd /project &&
cmake -B build
-DBUILD_SHARED_LIBS=ON
-DCMAKE_BUILD_TYPE=Release
-DSSLAUTHENTICATION=ON
-DNO_JAVA_WRAPPER=ON
-DENABLE_CAPNP=ON -DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r /usr/local/python_installer/* /project/source/wrappers/python/

- name: build uda on macos
if: runner.os == 'macOS'
run: >
brew update-reset && brew install
git
boost
openssl
cmake
libxml2
spdlog
capnp &&
cd ${{github.workspace}} &&
cmake -B build
-DBUILD_SHARED_LIBS=ON
-DCMAKE_BUILD_TYPE=Release
-DSSLAUTHENTICATION=ON
-DENABLE_CAPNP=ON
-DNO_JAVA_WRAPPER=ON
-DCMAKE_INSTALL_PREFIX=$PWD/install
-DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r $PWD/install/python_installer/* ${{github.workspace}}/source/wrappers/python/

- name: Build macos wheels
if: runner.os == 'macOS'
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_PLATFORM: macos
CIBW_BUILD: cp*-${{matrix.build-platform[2]}}

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

upload_pypi:
needs: build_wheels
runs-on: ubuntu-latest
environment:
name: testpypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true

- uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
163 changes: 163 additions & 0 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
name: build and upload wheels

on:
push:
tags:
- '*'

jobs:
build_wheels:
name: build wheels on ${{matrix.build-platform[0]}} ${{matrix.build-platform[1]}}
runs-on: ${{matrix.build-platform[0]}}
strategy:
fail-fast: false
matrix:
build-platform:
- [ubuntu-latest, x86_64, manylinux2014_x86_64]
- [ubuntu-latest, x86_64, manylinux_2_28_x86_64]
- [ubuntu-latest,aarch64,manylinux_2_28_aarch64]
- [macos-13, x86_64, macosx_x86_64]
- [macos-14, arm64, macosx_arm64]

steps:
- uses: actions/checkout@v3
# need git tags available for setuptools_scm to grab tags? will elad to slower checkout
with:
fetch-depth: 0

- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
with:
platforms: all

- name: Build manylinux2014 wheels
if: startswith(matrix.build-platform[2], 'manylinux2014')
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_BUILD: cp*-manylinux*
CIBW_SKIP: cp*-musllinux* *-musllinux*
CIBW_BEFORE_ALL: >
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* &&
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* &&
yum update -y &&
yum install -y wget openssl-devel libxml2-devel libtirpc-devel &&
cd /tmp &&
wget https://github.com/fmtlib/fmt/archive/refs/tags/10.0.0.tar.gz &&
tar xzf 10.0.0.tar.gz &&
cd fmt-10.0.0 &&
cmake -Bbuild -H. -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build -j --config Release --target install &&
cd /tmp &&
wget https://github.com/gabime/spdlog/archive/refs/tags/v1.11.0.tar.gz &&
tar xzf v1.11.0.tar.gz &&
cd spdlog-1.11.0 &&
cmake -Bbuild -H. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build -j --config Release --target install &&
cd /tmp &&
wget https://github.com/capnproto/capnproto/archive/refs/tags/v0.10.4.tar.gz &&
tar xzf v0.10.4.tar.gz &&
cd capnproto-0.10.4 &&
cmake -Bbuild -H. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON &&
cmake --build build &&
cmake --install build &&
cd /tmp &&
wget https://boostorg.jfrog.io/artifactory/main/release/1.80.0/source/boost_1_80_0.tar.gz &&
tar xzf boost_1_80_0.tar.gz &&
cd boost_1_80_0 &&
./bootstrap.sh --prefix=/usr/local &&
./b2 --without-python --prefix=/usr/local install &&
cd /project &&
cmake -B build -DNO_JAVA_WRAPPER=ON -DBUILD_SHARED_LIBS=ON -DSSLAUTHENTICATION=ON -DENABLE_CAPNP=ON -DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r /usr/local/python_installer/* /project/source/wrappers/python/

- name: Build manylinux_2_28 wheels
if: startswith(matrix.build-platform[2], 'manylinux_2_28')
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_28
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_BUILD: cp*-manylinux*
CIBW_SKIP: cp*-musllinux* *-musllinux*
CIBW_BEFORE_ALL: >
dnf update -y &&
dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm &&
dnf install -y boost-devel openssl-devel libxml2-devel libtirpc-devel
fmt fmt-devel spdlog spdlog-devel capnproto capnproto-devel &&
cd /project &&
cmake -B build
-DBUILD_SHARED_LIBS=ON
-DCMAKE_BUILD_TYPE=Release
-DSSLAUTHENTICATION=ON
-DNO_JAVA_WRAPPER=ON
-DENABLE_CAPNP=ON -DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r /usr/local/python_installer/* /project/source/wrappers/python/

- name: build uda on macos
if: runner.os == 'macOS'
run: >
brew update-reset && brew install
git
boost
openssl
cmake
libxml2
spdlog
capnp &&
cd ${{github.workspace}} &&
cmake -B build
-DBUILD_SHARED_LIBS=ON
-DCMAKE_BUILD_TYPE=Release
-DSSLAUTHENTICATION=ON
-DENABLE_CAPNP=ON
-DNO_JAVA_WRAPPER=ON
-DCMAKE_INSTALL_PREFIX=$PWD/install
-DCLIENT_ONLY=ON &&
cmake --build build -j --config Release --target install &&
cp -r $PWD/install/python_installer/* ${{github.workspace}}/source/wrappers/python/

- name: Build macos wheels
if: runner.os == 'macOS'
uses: pypa/[email protected]
with:
package-dir: ./source/wrappers/python
config-file: ./source/wrappers/python/pyproject.toml
env:
CIBW_ARCHS: ${{matrix.build-platform[1]}}
CIBW_PLATFORM: macos
CIBW_BUILD: cp*-${{matrix.build-platform[2]}}

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

upload_pypi:
needs: build_wheels
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/uda
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true

- uses: pypa/gh-action-pypi-publish@release/v1
2 changes: 1 addition & 1 deletion .github/workflows/cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ jobs:
python3 -m venv ${{github.workspace}}/venv &&
source ${{github.workspace}}/venv/bin/activate &&
pip3 install --upgrade pip &&
pip3 install wheel Cython numpy six &&
pip3 install wheel Cython "numpy<2" six &&
pip3 install ${{github.workspace}}/python_installer

- name: Test pyuda import
Expand Down
21 changes: 5 additions & 16 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,23 +45,12 @@ set( Boost_USE_MULTITHREADED OFF )
########################################################################################################################
# Version and Machine information

include( GetGitRevisionDescription )
# Set VERSION and FULL_VERSION from `git describe`
# but use git export attribute for release tarballs
set( GIT_ARCHIVE_DESCRIBE [[$Format:%(describe)$]] )
include( DetermineVersion )

git_describe( GIT_TAG --tags )

if( "${GIT_TAG}" MATCHES "^.*NOTFOUND$" )
message( WARNING "Failed to get git revision: ${GIT_TAG}" )
set( GIT_VERSION "0.0.0" )
elseif( "${GIT_TAG}" MATCHES "^([0-9]+\\.[0-9]+\\.[0-9]+)$" )
set( GIT_VERSION "${GIT_TAG}" )
elseif( "${GIT_TAG}" MATCHES "^([0-9]+\\.[0-9]+\\.[0-9]+)-([0-9]+)-.*$" )
set( GIT_VERSION "${CMAKE_MATCH_1}.${CMAKE_MATCH_2}" )
else()
message( WARNING "invalid git tag for version parsing: ${GIT_TAG}" )
set( GIT_VERSION "0.0.0" )
endif()

project( uda VERSION ${GIT_VERSION} )
project( uda VERSION ${VERSION} )

set( USER $ENV{USER} )

Expand Down
Loading
Loading