Building OSS Teleport from source to work around glibc version requirements

[webvictim]

If you're experiencing an error like teleport: /lib/libc.so.6: version 'GLIBC_2.28' not found (required by teleport) when trying to run Teleport, this means that your running distro does not provide a high enough version of glibc for the pre-compiled Teleport binaries to run. This is often the case with older distros like CentOS 6/7.

We have an open issue to look at reducing the version of glibc required for pre-compiled Teleport binaries, but in the meantime, to work around this issue, you could always look at compiling the open source version of Teleport yourself from source. This compiled version of Teleport will require the same version of glibc that your running distro does - you can see this version using ldd --version. It will also work on the same architecture that the running distro uses, so if you require arm or arm64 binaries, make sure to compile on a machine which also uses this architecture.

To compile Teleport, you'll need Go 1.15+ installed and available in your PATH, as well as gcc, make and zip. You can install these tools using package management:

CentOS/Fedora/RHEL: sudo yum -y install gcc make zip
Ubuntu/Debian: sudo apt-get -y install gcc make zip

Once these tools are installed, you can compile Teleport using these instructions:

git clone https://github.com/gravitational/teleport
cd teleport
git checkout v5.1.2 # update this to whatever version of Teleport you'd like to compile
make release

This will place a .tar.gz file in the local teleport directory, for example teleport-v5.1.2-linux-amd64-bin.tar.gz.

Other notes:

• any binaries compiled in this way:
  • won't have support for PAM or BPF, unless you also install those headers
  • can't be used to run an Enterprise version of the Teleport auth server (to use RBAC/SAML/OIDC functionality for example) as they compile the OSS version
    • you can connect OSS Teleport nodes to an Enterprise auth server with no problems, however.

[cshabazian]

Tried to do this so I can install on CentOS 7, but it won't compile. I have go (verified by compiling and running hello world program) and the other dependencies. This is a new build of Centos 7 with nothing else on it, and I checkout v8.0.0 (I also tried v7.0.0 and 5.1.2 to check) :

[root@localhost teleport]# make release
---> Building with GOOS=linux GOARCH=amd64 REPRODUCIBLE= and without PAM support and without FIPS support and without BPF support and without access tester and without Windows RDP client.
make --no-print-directory release-unix
---> Cleaning up OSS build artifacts.
rm -rf build
rm -rf
rm -rf
rm -rf lib/srv/desktop/rdp/rdpclient/target
rm -rf lib/datalog/roletester/target
go clean -cache
rm -rf teleport
rm -rf .gz
rm -rf .zip
rm -f gitref.go
echo "init webassets submodule"
init webassets submodule
git submodule update --init webassets
Submodule 'webassets' (https://github.com/gravitational/webassets.git) registered for path 'webassets'
Cloning into 'webassets'...
remote: Enumerating objects: 681, done.
remote: Counting objects: 100% (441/441), done.
remote: Compressing objects: 100% (320/320), done.
remote: Total 681 (delta 269), reused 292 (delta 121), pack-reused 240
Receiving objects: 100% (681/681), 10.23 MiB | 6.88 MiB/s, done.
Resolving deltas: 100% (383/383), done.
Submodule path 'webassets': checked out 'a1039e35e86aec770db6cdb32321c93356477757'
mkdir -p lib/web/build
---> Copying OSS web assets.
make all WEBASSETS_TAG="webassets_embed"
VERSION=8.0.0 make -f version.mk setver
for CHART in teleport-cluster teleport-kube-agent; do
sed -i'.bak' -e "s_^version:\ ._version: "${VERSION}"g" examples/chart/${CHART}/Chart.yaml || exit 1;
sed -i'.bak' -e "s^appVersion:\ .appVersion: "${VERSION}"g" examples/chart/${CHART}/Chart.yaml || exit 1;
rm -f examples/chart/${CHART}/Chart.yaml.bak;
done
---> Building OSS binaries.
make build/teleport build/tctl build/tsh
GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -tags " webassets_embed " -o build/teleport -ldflags '-w -s' ./tool/teleport
# runtime/internal/sys
/usr/local/go/src/runtime/internal/sys/stubs.go:9:7: PtrSize redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:24:38: previous declaration
/usr/local/go/src/runtime/internal/sys/stubs.go:10:24: undefined: Uintreg
/usr/local/go/src/runtime/internal/sys/stubs.go:16:7: StackGuardMultiplier redeclared in this block
/usr/local/go/src/runtime/internal/sys/arch.go:27:74: previous declaration
# runtime/internal/atomic
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:18:6: Load redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:16:24: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:24:6: Loadp redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:22:32: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:30:6: Load64 redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:28:26: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:36:6: LoadAcq redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:34:27: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:41:6: Xadd redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:51:37: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:44:6: Xadd64 redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:54:39: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:47:6: Xadduintptr redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:57:47: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:50:6: Xchg redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:60:36: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:53:6: Xchg64 redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:63:38: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: Xchguintptr redeclared in this block
/usr/local/go/src/runtime/internal/atomic/atomic_amd64.go:66:45: previous declaration
/usr/local/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: too many errors
make[3]: *** [build/teleport] Error 2
make[2]: *** [all] Error 2
make[1]: *** [full] Error 2
make: *** [release] Error 2

[webvictim]

It worked fine for me - here's a log so you can see what I ran:

~ » docker run -it centos:7 /bin/bash
[root@fd184d0949b4 /]# yum -y update && yum -y install wget zip make gcc git
Loaded plugins: fastestmirror, ovl
<snip>
Complete!
[root@fd184d0949b4 /]# cd
[root@fd184d0949b4 ~]# wget https://golang.org/dl/go1.17.3.linux-amd64.tar.gz
--2021-11-18 14:53:51--  https://golang.org/dl/go1.17.3.linux-amd64.tar.gz
Resolving golang.org (golang.org)... 142.250.80.81, 2607:f8b0:4006:80c::2011
Connecting to golang.org (golang.org)|142.250.80.81|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://dl.google.com/go/go1.17.3.linux-amd64.tar.gz [following]
--2021-11-18 14:53:51--  https://dl.google.com/go/go1.17.3.linux-amd64.tar.gz
Resolving dl.google.com (dl.google.com)... 142.250.80.46, 2607:f8b0:4006:80b::200e
Connecting to dl.google.com (dl.google.com)|142.250.80.46|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 134804820 (129M) [application/x-gzip]
Saving to: 'go1.17.3.linux-amd64.tar.gz'

100%[=================================================================================================================================================================>] 134,804,820 55.1MB/s   in 2.3s

2021-11-18 14:53:53 (55.1 MB/s) - 'go1.17.3.linux-amd64.tar.gz' saved [134804820/134804820]

[root@fd184d0949b4 ~]# tar -C /usr/local -xvzf go1.17.3.linux-amd64.tar.gz
<snip>
[root@fd184d0949b4 ~]# export PATH=/usr/local/go/bin:$PATH
[root@fd184d0949b4 ~]# go version
go version go1.17.3 linux/amd64
[root@fd184d0949b4 ~]# git clone https://github.com/gravitational/teleport
Cloning into 'teleport'...
remote: Enumerating objects: 124187, done.
remote: Counting objects: 100% (6610/6610), done.
remote: Compressing objects: 100% (3018/3018), done.
remote: Total 124187 (delta 4241), reused 5473 (delta 3477), pack-reused 117577
Receiving objects: 100% (124187/124187), 192.71 MiB | 3.48 MiB/s, done.
Resolving deltas: 100% (84582/84582), done.
[root@fd184d0949b4 ~]# cd teleport
[root@fd184d0949b4 teleport]# git checkout v8.0.0
Note: checking out 'v8.0.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 6720c97... Release 8.0.0.
[root@fd184d0949b4 teleport]# make release
go: downloading google.golang.org/grpc v1.27.0
go: downloading github.com/gravitational/trace v1.1.15
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/sirupsen/logrus v1.6.0
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/golang/protobuf v1.4.2
go: downloading golang.org/x/net v0.0.0-20200707034311-ab3426394381
go: downloading github.com/google/go-cmp v0.5.4
go: downloading golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd
go: downloading github.com/jonboulle/clockwork v0.2.2
go: downloading google.golang.org/protobuf v1.23.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading golang.org/x/text v0.3.0
---> Building with GOOS=linux GOARCH=amd64 REPRODUCIBLE= and without PAM support and without FIPS support and without BPF support and without access tester and without Windows RDP client.
make --no-print-directory release-unix
---> Cleaning up OSS build artifacts.
rm -rf build
rm -rf
rm -rf
rm -rf lib/srv/desktop/rdp/rdpclient/target
rm -rf lib/datalog/roletester/target
go clean -cache
rm -rf teleport
rm -rf *.gz
rm -rf *.zip
rm -f gitref.go
echo "init webassets submodule"
init webassets submodule
git submodule update --init webassets
Submodule 'webassets' (https://github.com/gravitational/webassets.git) registered for path 'webassets'
Cloning into 'webassets'...
remote: Enumerating objects: 681, done.
remote: Counting objects: 100% (441/441), done.
remote: Compressing objects: 100% (320/320), done.
remote: Total 681 (delta 269), reused 292 (delta 121), pack-reused 240
Receiving objects: 100% (681/681), 10.23 MiB | 9.89 MiB/s, done.
Resolving deltas: 100% (383/383), done.
Submodule path 'webassets': checked out 'a1039e35e86aec770db6cdb32321c93356477757'
mkdir -p lib/web/build
---> Copying OSS web assets.
make all WEBASSETS_TAG="webassets_embed"
VERSION=8.0.0 make -f version.mk setver
for CHART in teleport-cluster teleport-kube-agent; do \
    sed -i'.bak' -e "s_^version:\ .*_version: \"${VERSION}\"_g" examples/chart/${CHART}/Chart.yaml || exit 1; \
    sed -i'.bak' -e "s_^appVersion:\ .*_appVersion: \"${VERSION}\"_g" examples/chart/${CHART}/Chart.yaml || exit 1; \
    rm -f examples/chart/${CHART}/Chart.yaml.bak; \
done
---> Building OSS binaries.
make build/teleport build/tctl build/tsh
GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -tags "   webassets_embed " -o build/teleport  -ldflags '-w -s' ./tool/teleport
GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -tags "   " -o build/tctl  -ldflags '-w -s' ./tool/tctl
GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -tags " " -o build/tsh  -ldflags '-w -s' ./tool/tsh
---> Creating OSS release archive.
mkdir teleport
cp -rf build/* \
    examples \
    build.assets/install\
    README.md \
    CHANGELOG.md \
    teleport/
echo v8.0.0 > teleport/VERSION
tar  -c teleport | gzip -n > teleport-v8.0.0-linux-amd64-bin.tar.gz
rm -rf teleport
---> Created teleport-v8.0.0-linux-amd64-bin.tar.gz.
[root@fd184d0949b4 teleport]# ./build/teleport version
Teleport v8.0.0 git:v8.0.0-0-g6720c97 go1.17.3

[webvictim]

I realise this doesn't answer your question directly, but you can run the pre-compiled CentOS 6 binaries on CentOS 7 without issue: https://get.gravitational.com/teleport-v8.0.0-linux-amd64-centos6-bin.tar.gz for 8.0.0 for example.

You can get other versions at https://goteleport.com/teleport/download/

[cshabazian]

Ok I got it to run. There is an additional dependency. I didn't figure out what it is, I just add the development tools group:
yum groupinstall "Development Tools"
and it added whatever was missing.

Thanks for the help.

[gknw]

This way also works for building packages for old Debian systems. Tested on Debian 7, 8 and 9. On Debian 7 and Debian 8 you must update binutils first, because an ld errors. See below for the script that can be running inside docker container based on debain:7 image.

#!/bin/bash

set -eu

declare -r VERSION="${1:-8.0.6}"
declare -r DEBIAN_VERSION="${2:-7}"
declare -r GO_VERSION="${3:-1.17.5}"
declare -r GIT_REPO_DIR="${4:-./git}"


mkdir -p "${GIT_REPO_DIR}"

sed -i 's|deb.debian.org|archive.debian.org|g' /etc/apt/sources.list \
    && sed -i '/updates/d' /etc/apt/sources.list \
    && apt-get update \
    && apt-get upgrade -y

apt-get install -y --force-yes \
    wget curl ca-certificates libgmp-dev libmpfr-dev libmpc-dev \
    gcc make bison texinfo flex build-essential \
    git perl-modules libswitch-perl perl perl-base=5.14.2-21+deb7u3 \
    libc6-dev=2.13-38+deb7u10 libc6=2.13-38+deb7u10 libc-bin=2.13-38+deb7u10 libc-dev-bin=2.13-38+deb7u10

# PAM support
apt-get install -y libpam0g-dev

wget "https://golang.org/dl/go${GO_VERSION}.linux-amd64.tar.gz" -qO - | tar xzf - -C /usr/local

export PATH="/usr/local/go/bin:$PATH"
go version

git clone https://github.com/gravitational/teleport.git "${GIT_REPO_DIR}/teleport"
git clone git://sourceware.org/git/binutils-gdb.git "${GIT_REPO_DIR}/binutils-gdb"

cd "${GIT_REPO_DIR}/binutils-gdb" \
    && git checkout binutils-2_26_1 \
    && ./configure \
    && make \
    && make install \
    && cd - 

cd "${GIT_REPO_DIR}/teleport" \
    && git checkout v${VERSION}\
    && make release \
    && cd -