Installation Steps

[MuffinsThaCat]

The step we're following to install are here : https://github.com/edgelesssys/marblerun/blob/master/BUILD.md#build

/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2```

[daniel-weisse]

Hi there,
Can you please provide the following information:

• Your OS, e.g. from cat /etc/os-release
• Go version, e.g. from go version
• EdgelessRT version, e.g. from apt show edglessrt
• The git commit you are trying to build from, e.g. from git rev-parse HEAD
• Are you to following the Docker instructions or the local building instructions ?

[dhruvinparikh]

Hi there, Can you please provide the following information:

• Your OS, e.g. from cat /etc/os-release
• Go version, e.g. from go version
• EdgelessRT version, e.g. from apt show edglessrt
• The git commit you are trying to build from, e.g. from git rev-parse HEAD
• Are you to following the Docker instructions or the local building instructions ?

• cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

• go version

go version go1.21.2 linux/arm64

• apt show edglessrt

N: Unable to locate package edglessrt
N: Unable to locate package edglessrt
E: No packages found

• git rev-parse HEAD

c33669b10382afad721d2dd534504dbcfff125df

• Having issues setting up environment itself. But I am able to build using docker instructions with no issues.

[daniel-weisse]

Hi @dhruvinparikh
Please follow the installation instructions for EdglessRT to set up the prerequisites for building MarbleRun.

[MuffinsThaCat]

It was installed but not sure why it isn't showing up in apt, I think was the issue. Looking into it. Will update

[MuffinsThaCat]

When we build the Coordinator control plane and Marble test applications:

mkdir build
cd build
cmake ..
make

I am stuck at make with the following error:

runtime/cgo

gcc_arm64.S: Assembler messages:
gcc_arm64.S:30: Error: no such instruction: stp x29,x30,[sp,' gcc_arm64.S:34: Error: too many memory references for mov'
gcc_arm64.S:36: Error: no such instruction: stp x19,x20,[sp,' gcc_arm64.S:39: Error: no such instruction: stp x21,x22,[sp,'
gcc_arm64.S:42: Error: no such instruction: stp x23,x24,[sp,' gcc_arm64.S:45: Error: no such instruction: stp x25,x26,[sp,'
gcc_arm64.S:48: Error: no such instruction: stp x27,x28,[sp,' gcc_arm64.S:52: Error: too many memory references for mov'
gcc_arm64.S:53: Error: too many memory references for mov' gcc_arm64.S:54: Error: too many memory references for mov'
gcc_arm64.S:56: Error: no such instruction: blr x20' gcc_arm64.S:57: Error: no such instruction: blr x19'
gcc_arm64.S:59: Error: no such instruction: ldp x27,x28,[sp,' gcc_arm64.S:62: Error: no such instruction: ldp x25,x26,[sp,'
gcc_arm64.S:65: Error: no such instruction: ldp x23,x24,[sp,' gcc_arm64.S:68: Error: no such instruction: ldp x21,x22,[sp,'
gcc_arm64.S:71: Error: no such instruction: ldp x19,x20,[sp,' gcc_arm64.S:74: Error: no such instruction: ldp x29,x30,[sp],'
[ 0%] Built target coordinator-noenclave
[ 0%] Built target coordinatorlib
Consolidate compiler generated dependencies of target coordinator-enclave
[ 11%] Linking CXX executable coordinator-enclave
[ 22%] Built target coordinator-enclave
[ 33%] Built target signing-key
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

[daniel-weisse]

gcc_arm64.S: Assembler messages:

Are you trying to build MarbleRun on an ARM host?
Or do you have the GOARCH environment variable set to arm64?

In the second case, unset the GOARCH env variable, and try again.
In the first case, this will not work out of the box.
You might be successful if you just export GOARCH=amd64, but will likely also have to tell the build script to use a toolchain that targets amd64 hosts (using the CC env variable).
But I'm not confident enough to tell you that this will work, as cross compiling for different architectures can be a bit tricky, especially if CGO is involved.

[MuffinsThaCat]

uname -a
Linux d1cc01526652 6.6.12-linuxkit #1 SMP Fri Jan 19 08:53:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

I tried following unset GOARCH make but still same error

[ 0%] Built target coordinator-noenclave
[ 0%] Built target coordinatorlib
[ 22%] Built target coordinator-enclave
[ 33%] Built target signing-key
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

[thomasten]

Hi, can you please provide the full terminal output of all these commands

. /opt/edgelessrt/share/openenclave/openenclaverc
mkdir build && cd build
cmake ..
OE_LOG_LEVEL=INFO make VERBOSE=1
ls -l

[MuffinsThaCat]

root@d1cc01526652:/com.docker.devenvironments.code/marblerun# mkdir build && cd build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:4 (find_package):
Could not find a package configuration file provided by "OpenEnclave" with
any of the following names:

 OpenEnclaveConfig.cmake
 openenclave-config.cmake

Add the installation prefix of "OpenEnclave" to CMAKE_PREFIX_PATH or set
"OpenEnclave_DIR" to a directory containing one of the above files. If
"OpenEnclave" provides a separate development package or SDK, be sure it
has been installed.

-- Configuring incomplete, errors occurred!
See also "/com.docker.devenvironments.code/marblerun/build/CMakeFiles/CMakeOutput.log".
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# OE_LOG_LEVEL=INFO make VERBOSE=1
make: *** No targets specified and no makefile found. Stop.
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# ls -l
total 16
-rw-r--r-- 1 root root 14292 Feb 16 14:34 CMakeCache.txt
drwxr-xr-x 6 root root 192 Feb 16 14:34 CMakeFiles
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build#

[thomasten]

updated my comment to include the source command

[MuffinsThaCat]

Works, thanks!

[MuffinsThaCat]

Actually sorry, need to unclose this for a second. There was an error at the end we missed.

Updated terminal log
root@d1cc01526652:/com.docker.devenvironments.code/marblerun# mkdir build && cd build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for crypto library - found
-- Looking for dl library - found
CMake Warning at /opt/edgelessrt/lib/openenclave/cmake/openenclave-config.cmake:122 (message):
-- Looking for sgx_dcap_ql library - not found. Attestations based on
quotes would not function without the quote provider.
Call Stack (most recent call first):
CMakeLists.txt:4 (find_package)
-- Performing Test OE_SPECTRE_MITIGATION_C_FLAGS_SUPPORTED
-- Performing Test OE_SPECTRE_MITIGATION_C_FLAGS_SUPPORTED - Failed
-- Performing Test OE_SPECTRE_MITIGATION_CXX_FLAGS_SUPPORTED
-- Performing Test OE_SPECTRE_MITIGATION_CXX_FLAGS_SUPPORTED - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /com.docker.devenvironments.code/marblerun/build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# OE_LOG_LEVEL=INFO make VERBOSE=1
/usr/bin/cmake -S/com.docker.devenvironments.code/marblerun -B/com.docker.devenvironments.code/marblerun/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /com.docker.devenvironments.code/marblerun/build/CMakeFiles /com.docker.devenvironments.code/marblerun/build//CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-noenclave.dir/build.make CMakeFiles/coordinator-noenclave.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinator-noenclave.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-noenclave.dir/build.make CMakeFiles/coordinator-noenclave.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/cmd/coordinator && /usr/bin/cmake -P /com.docker.devenvironments.code/marblerun/build_with_version.cmake go 1.4.0 /com.docker.devenvironments.code/marblerun/build/coordinator-noenclave main
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 0%] Built target coordinator-noenclave
make -f CMakeFiles/coordinatorlib.dir/build.make CMakeFiles/coordinatorlib.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinatorlib.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinatorlib.dir/build.make CMakeFiles/coordinatorlib.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/cmd/coordinator && /usr/bin/cmake -P /com.docker.devenvironments.code/marblerun/build_with_version.cmake ertgo 1.4.0 /com.docker.devenvironments.code/marblerun/build/libcoordinator.a main
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 0%] Built target coordinatorlib
make -f CMakeFiles/coordinator-enclave.dir/build.make CMakeFiles/coordinator-enclave.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinator-enclave.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-enclave.dir/build.make CMakeFiles/coordinator-enclave.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 11%] Building C object CMakeFiles/coordinator-enclave.dir/enclave/main.c.o
/usr/bin/cc -DOE_BUILD_ENCLAVE -isystem /opt/edgelessrt/include -isystem /opt/edgelessrt/include/openenclave/3rdparty/libc -isystem /opt/edgelessrt/include/openenclave/3rdparty -g -m64 -fPIE -nostdinc -fstack-protector-strong -fvisibility=hidden -fno-omit-frame-pointer -ffunction-sections -fdata-sections -ftls-model=local-exec -MD -MT CMakeFiles/coordinator-enclave.dir/enclave/main.c.o -MF CMakeFiles/coordinator-enclave.dir/enclave/main.c.o.d -o CMakeFiles/coordinator-enclave.dir/enclave/main.c.o -c /com.docker.devenvironments.code/marblerun/enclave/main.c
[ 22%] Linking CXX executable coordinator-enclave
/usr/bin/cmake -E cmake_link_script CMakeFiles/coordinator-enclave.dir/link.txt --verbose=1
/usr/bin/c++ -g CMakeFiles/coordinator-enclave.dir/enclave/main.c.o /opt/edgelessrt/lib/openenclave/enclave/objects-Release/ertmeshentry/meshentry.cpp.o /opt/edgelessrt/lib/openenclave/enclave/objects-Release/ertmeshentry/emain_t.c.o -o coordinator-enclave /opt/edgelessrt/lib/openenclave/enclave/liboeenclave.a libcoordinator.a /opt/edgelessrt/lib/openenclave/enclave/libertttls.a /opt/edgelessrt/lib/openenclave/enclave/libertlibc.a -static -Wl,--eh-frame-hdr -Wl,-u,ert_syscall -lstdc++ /opt/edgelessrt/lib/openenclave/enclave/libertlibunwind.a /opt/edgelessrt/lib/openenclave/enclave/liboecryptombedtls.a /opt/edgelessrt/lib/openenclave/enclave/libmbedtls.a /opt/edgelessrt/lib/openenclave/enclave/libmbedx509.a /opt/edgelessrt/lib/openenclave/enclave/libmbedcrypto.a /opt/edgelessrt/lib/openenclave/enclave/liboehostepoll.a /opt/edgelessrt/lib/openenclave/enclave/liboehostfs.a /opt/edgelessrt/lib/openenclave/enclave/liboehostresolver.a /opt/edgelessrt/lib/openenclave/enclave/liboehostsock.a /opt/edgelessrt/lib/openenclave/enclave/liboelibc.a /opt/edgelessrt/lib/openenclave/enclave/liboesyscall.a /opt/edgelessrt/lib/openenclave/enclave/liboelibc.a /opt/edgelessrt/lib/openenclave/enclave/liboesyscall.a /opt/edgelessrt/lib/openenclave/enclave/liboecore.a -nostdlib -nodefaultlibs -nostartfiles -Wl,--no-undefined,-Bstatic,-Bsymbolic,--export-dynamic,-pie,--build-id -Wl,-z,noexecstack -Wl,-z,now -Wl,-gc-sections
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 22%] Built target coordinator-enclave
make -f CMakeFiles/signing-key.dir/build.make CMakeFiles/signing-key.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/signing-key.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/signing-key.dir/build.make CMakeFiles/signing-key.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 33%] Generating private.pem, public.pem
openssl genrsa -out private.pem -3 3072
openssl rsa -in private.pem -pubout -out public.pem
writing RSA key
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 33%] Built target signing-key
make -f CMakeFiles/sign-coordinator.dir/build.make CMakeFiles/sign-coordinator.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/sign-coordinator.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/sign-coordinator.dir/build.make CMakeFiles/sign-coordinator.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign eradump -e coordinator-enclave.signed > coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make[1]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make: *** [Makefile:91: all] Error 2
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# ls -l
total 76048
-rw-r--r-- 1 root root 14994 Feb 16 17:41 CMakeCache.txt
drwxr-xr-x 26 root root 832 Feb 16 17:41 CMakeFiles
-rw-r--r-- 1 root root 1665 Feb 16 17:41 cmake_install.cmake
-rw-r--r-- 1 root root 107 Feb 16 17:41 coordinator.conf
-rwxr-xr-x 1 root root 34408432 Feb 16 17:41 coordinator-enclave
-rw-r--r-- 1 root root 43300922 Feb 16 17:41 libcoordinator.a
-rw-r--r-- 1 root root 1758 Feb 16 17:41 libcoordinator.h
-rw-r--r-- 1 root root 11399 Feb 16 17:41 Makefile
-rw------- 1 root root 2484 Feb 16 17:41 private.pem
-rw-r--r-- 1 root root 621 Feb 16 17:41 public.pem
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build#

The error: ' make: *** [Makefile:91: all] Error 2 '

It looks like we're close but something may be missing.

[thomasten]

It seems it doesn't (even try to) sign the coordinator-enclave binary. We have [ 22%] Built target coordinator-enclave and [ 44%] Generating coordinator-config.json. But between these two, Generating coordinator-enclave.signed is missing. What cmake version is this?