Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replacing openssl111 with openssl3 #94

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

replacing openssl111 with openssl3 #94

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
af1a401
changed minitest.py for one port
Oct 26, 2024
f1c6f3e
changed changed path of liboqs.so
Oct 26, 2024
208cb96
updated docker file to use openssl3
Oct 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 87 additions & 25 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,65 +1,127 @@
# Multi-stage build: First the full builder image:

# liboqs build type variant; maximum portability of image; no openssl dependency:
ARG LIBOQS_BUILD_DEFINES="-DOQS_DIST_BUILD=ON -DBUILD_SHARED_LIBS=ON -DOQS_USE_OPENSSL=OFF"
# define the liboqs tag to be used
ARG LIBOQS_TAG=main

# make build arguments: Adding -j here speeds up build but may tax hardware
ARG MAKE_DEFINES="-j 2"
# define the oqsprovider tag to be used
ARG OQSPROVIDER_TAG=main

FROM alpine:3.16 as intermediate
# Default location where all binaries wind up:
ARG INSTALLDIR=/opt/oqssa

# liboqs build type variant; maximum portability of image:
ARG LIBOQS_BUILD_DEFINES="-DOQS_DIST_BUILD=ON -DBUILD_SHARED_LIBS=ON"

# Default root CA signature algorithm; can be set to any listed at https://github.com/open-quantum-safe/oqs-provider#algorithms
ARG SIG_ALG="dilithium2"

# Default KEM algorithms; can be set to any listed at https://github.com/open-quantum-safe/oqs-provider#algorithms
ARG DEFAULT_GROUPS="x25519:x448:kyber512:p256_kyber512:kyber768:p384_kyber768:kyber1024:p521_kyber1024"

# Define the degree of parallelism when building the image; leave the number away only if you know what you are doing
ARG MAKE_DEFINES="-j 4"

ARG PYTHON_VERSION=3.11.0

FROM alpine:latest as intermediate
# Take in all global args
ARG LIBOQS_TAG
ARG OQSPROVIDER_TAG
ARG INSTALLDIR
ARG LIBOQS_BUILD_DEFINES
ARG SIG_ALG
ARG DEFAULT_GROUPS
ARG MAKE_DEFINES

LABEL version="2"
LABEL version="4"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be version 3.


ENV DEBIAN_FRONTEND noninteractive
ENV DEBIAN_FRONTEND=noninteractive

RUN apk update && apk upgrade

# Get all software packages required for builing all components:
RUN apk add build-base linux-headers cmake ninja git
RUN apk add build-base linux-headers \
libffi-dev zlib-dev xz-dev bzip2-dev \
libtool automake autoconf cmake ninja \
make \
openssl3 openssl3-dev \
git wget \
py3-pip python3

# get all sources
WORKDIR /opt
RUN git clone --depth 1 --branch main https://github.com/open-quantum-safe/liboqs && \
git clone --depth 1 --branch main https://github.com/open-quantum-safe/liboqs-python.git
RUN git clone --depth 1 --branch ${LIBOQS_TAG} https://github.com/open-quantum-safe/liboqs && \
git clone --depth 1 --branch master https://github.com/openssl/openssl.git && \
git clone --depth 1 --branch ${OQSPROVIDER_TAG} https://github.com/open-quantum-safe/oqs-provider.git && \
git clone https://github.com/gobbledy-gook/liboqs-python.git

# build liboqs

# build liboqs
WORKDIR /opt/liboqs
RUN mkdir build && cd build && cmake -GNinja .. ${LIBOQS_BUILD_DEFINES} && ninja install

RUN mkdir build && cd build && cmake -G"Ninja" .. ${LIBOQS_BUILD_DEFINES} -DCMAKE_INSTALL_PREFIX=${INSTALLDIR} && ninja install

# build OpenSSL3
WORKDIR /opt/openssl
RUN LDFLAGS="-Wl,-rpath -Wl,${INSTALLDIR}/lib64:${INSTALLDIR}/lib" ./config shared --prefix=${INSTALLDIR} && \
make ${MAKE_DEFINES} && make install_sw install_ssldirs;

# set path to use 'new' openssl. Dyn libs have been properly linked in to match
ENV PATH="${INSTALLDIR}/bin:${PATH}"

# build & install provider (and activate by default)
WORKDIR /opt/oqs-provider
RUN ln -s ../openssl . && cmake -DOPENSSL_ROOT_DIR=${INSTALLDIR} -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=${INSTALLDIR} -S . -B _build && cmake --build _build && cp _build/lib/oqsprovider.so ${INSTALLDIR}/lib64/ossl-modules && sed -i "s/default = default_sect/default = default_sect\noqsprovider = oqsprovider_sect/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\[default_sect\]/\[default_sect\]\nactivate = 1\n\[oqsprovider_sect\]\nactivate = 1\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/providers = provider_sect/providers = provider_sect\nssl_conf = ssl_sect\n\n\[ssl_sect\]\nsystem_default = system_default_sect\n\n\[system_default_sect\]\nGroups = \$ENV\:\:DEFAULT_GROUPS\n/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/\# Use this in order to automatically load providers/\# Set default KEM groups if not set via environment variable\nKDEFAULT_GROUPS = $DEFAULT_GROUPS\n\n# Use this in order to automatically load providers/g" /opt/oqssa/ssl/openssl.cnf && sed -i "s/HOME\t\t\t= ./HOME\t\t= .\nDEFAULT_GROUPS\t= ${DEFAULT_GROUPS}/g" /opt/oqssa/ssl/openssl.cnf

# Building Python using OPENSSL3
# WORKDIR /usr/local
# RUN wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz
# RUN tar -xzf Python-3.11.0.tgz
# RUN cd Python-3.11.0 && ./configure --with-openssl=${INSTALLDIR} --with-openssl-rpath=${INSTALLDIR}/lib64 --enable-optimizations && \
# make && make altinstall
# # Check where Python is installed
# RUN echo "Checking installed binaries:" && ls -l /usr/local/bin && ls -l /usr/bin
# generate certificates for openssl s_server, which is what we will test curl against
ENV OPENSSL=${INSTALLDIR}/bin/openssl
ENV OPENSSL_CONF=${INSTALLDIR}/ssl/openssl.cnf


# Download and integrate LetsEncrypt Root CA to CA bundle
WORKDIR /opt
RUN git clone --depth 1 --branch OQS-OpenSSL_1_1_1-stable https://github.com/open-quantum-safe/openssl.git && cd liboqs && mkdir build-openssl && cd build-openssl && cmake -G"Ninja" .. ${LIBOQS_BUILD_DEFINES} -DCMAKE_INSTALL_PREFIX=/opt/openssl/oqs && ninja install

RUN apk add automake autoconf && cd /opt/openssl && LDFLAGS="-Wl,-rpath -Wl,/usr/local/lib64" ./Configure shared linux-x86_64 -lm && make ${MAKE_DEFINES} && make install_sw

# Get LetsEncrypt root
RUN wget https://letsencrypt.org/certs/isrgrootx1.pem


## second stage: Only create minimal image without build tooling and intermediate build results generated above:
FROM alpine:3.16
FROM alpine:latest
# Take in all global args
ARG INSTALLDIR
ARG SIG_ALG

# Get all software packages required for running all components:
RUN apk update && apk upgrade && apk add python3
RUN apk update && apk upgrade
RUN apk add py3-pip python3

# Only retain the binary contents in the final image
# Only retain the ${INSTALLDIR} contents in the final image
COPY --from=intermediate ${INSTALLDIR} ${INSTALLDIR}
COPY --from=intermediate /usr/local /usr/local
COPY --from=intermediate /opt/liboqs-python /opt/liboqs-python

# set path to use 'new' openssl. Dyn libs have been properly linked in to match
ENV PATH="${INSTALLDIR}/bin:${PATH}"
ENV PYTHONPATH=/opt/liboqs-python

# generate certificates for openssl s_server, which is what we will test curl against
ENV OPENSSL=${INSTALLDIR}/bin/openssl
ENV OPENSSL_CONF=${INSTALLDIR}/ssl/openssl.cnf

# Install liboqs-python
RUN cd /opt/liboqs-python && python3 setup.py install

# Enable a normal user
RUN addgroup -g 1000 -S oqs && adduser --uid 1000 -S oqs -G oqs

USER oqs
WORKDIR /home/oqs
COPY minitest.py /home/oqs/minitest.py
COPY ./liboqs-python/docker/minitest.py /home/oqs/minitest.py
COPY --from=intermediate /opt/isrgrootx1.pem /home/oqs/isrgrootx1.pem

# ensure oqs libs are found. Unset if interested in using stock openssl:
ENV LD_LIBRARY_PATH=/usr/local/lib64
ENV LD_LIBRARY_PATH=${INSTALLDIR}/lib64
CMD ["python3", "minitest.py"]
48 changes: 31 additions & 17 deletions docker/minitest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,34 @@
# Trust test.openquantumsafe.org root CA:
sslContext.load_verify_locations(cafile="CA.crt")

# Iterate over all algorithm/port combinations:
for sigs, kexs in assignments.items():
for kex, port in kexs.items():
if (kex != "*"): # '*' denoting any classic KEX alg
# Enable use of the specific QSC KEX algorithm
os.environ["TLS_DEFAULT_GROUPS"]=kex
try:
with urllib.request.urlopen('https://test.openquantumsafe.org:'+str(port), context=sslContext) as response:
if response.getcode() != 200:
print("Failed to test %s successfully" % (kex))
else:
print("Success testing %s at port %d" % (kex, port))
except:
print("Test of algorithm combination SIG %s/KEX %s failed. Are all algorithms supported by current OQS library?" % (sigs, kex))

if "SHORT_TEST" in os.environ:
exit(0)
port = 6138
kex="kyber512"
sigs = "dilithium2"

try:
with urllib.request.urlopen('https://test.openquantumsafe.org:'+str(port), context=sslContext) as response:
if response.getcode() != 200:
print("Failed to test %s successfully" % (kex))
else:
print("Success testing %s at port %d" % (kex, port))
except:
print("Test of algorithm combination SIG %s/KEX %s failed. Are all algorithms supported by current OQS library?" % (sigs, kex))


# # Iterate over all algorithm/port combinations:
# for sigs, kexs in assignments.items():
# for kex, port in kexs.items():
# if (kex != "*"): # '*' denoting any classic KEX alg
# # Enable use of the specific QSC KEX algorithm
# os.environ["TLS_DEFAULT_GROUPS"]=kex
# try:
# with urllib.request.urlopen('https://test.openquantumsafe.org:'+str(port), context=sslContext) as response:
# if response.getcode() != 200:
# print("Failed to test %s successfully" % (kex))
# else:
# print("Success testing %s at port %d" % (kex, port))
# except:
# print("Test of algorithm combination SIG %s/KEX %s failed. Are all algorithms supported by current OQS library?" % (sigs, kex))

# if "SHORT_TEST" in os.environ:
# exit(0)
4 changes: 2 additions & 2 deletions oqs/oqs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,8 @@ def _install_liboqs(target_directory, oqs_version=None):


def _load_liboqs():
home_dir = os.path.expanduser("~")
oqs_install_dir = os.path.abspath(home_dir + os.path.sep + "_oqs") # $HOME/_oqs
home_dir = os.path.expanduser("/opt")
oqs_install_dir = os.path.abspath(home_dir + os.path.sep + "oqssa") # $HOME/_oqs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this change is causing the CI failures—is it necessary?

Copy link
Author

@gobbledy-gook gobbledy-gook Nov 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this is utterly necessary or not, but it is where the shared object liboqs.so is found by oqs.py.

As I understand (please correct if I am wrong), previously, liboqs.so was being installed during runtime (while oqs.py tries to load the shared object but does not find it and as a result installs at ~/_oqs), however in this case the shared object is required to build ops-provider. Thus now it is installed in ${INSTALLDIR}=/opt/oqssa with all other binaries.

oqs_lib_dir = (
os.path.abspath(oqs_install_dir + os.path.sep + "bin") # $HOME/_oqs/bin
if platform.system() == "Windows"
Expand Down
Loading