The machinekit package build and distribution process

machinekit uses the Travis service for continuous integration and package builds.

Flow - in a nutshell:

1. A pull request is posted, to or merged into https://github.com/machinekit/machinekit

2. this PR triggers the travis integration as defined here, using these supporting scripts.

3. packages are built for a merge only; for a mere pull request, only regression tests are run.

4. If regressions pass, and the debian packages build successfully, packages uploaded from travis to the distribution host and (currently) to packagecloud.io

5. On these hosts, packages are signed and made available for download via apt-get.

Setup on the distribution host:

There are four docker containers controlled by https://github.com/machinekit/mk_deb_server/blob/master/docker-compose.yml, named data, sftp, reprepro and web

to view them, run “docker ps”:

56a8a3392d4f        nginx                                     "nginx -g 'daemon off"   5 weeks ago         Up 5 weeks          0.0.0.0:9080->80/tcp, 0.0.0.0:9443->443/tcp   mk_web_server
b8daa26cafba        mkdebs_reprepro                           "/bin/sh -c /run.sh"     5 weeks ago         Up 15 minutes                                                     mk_reprepro
d98a875452b1        mkdebs_sftp                               "/bin/sh -c '/run.sh "   5 weeks ago         Up 5 weeks          0.0.0.0:9022->22/tcp                          mk_sftp_server

data is essentially being a folder, not a process - just a container for storing data that is being shared between the 3 containers

• mkdebs_sftp runs the sftp server accepting incoming packages from travis, listening on port 9022 - see matching travis config

• mkdebs_reprepro signs the packages and inserts the packages into the repo hierarchy

• nginx serves http requests from apt-get

The docker containers start automatically after a reboot because of these lines: https://github.com/kinsamanka/mkdebs/blob/master/docker-compose.yml#L8 https://github.com/kinsamanka/mkdebs/blob/master/docker-compose.yml#L20 https://github.com/kinsamanka/mkdebs/blob/master/docker-compose.yml#L34

as long as docker starts on reboot, the containers will be started automatically

IP address setup on the distribution host (assuming shorewall):

Docker containers live in the dock zone, virtual interface docker0, 172.17.0.1/16. The public IP address used for the sftp and nginx containers: 86.59.12.252 on eth0 .

To make the web and sftp container reachable, masquerading is used:

mah2:/etc/shorewall# cat masq
#INTERFACE:DEST         SOURCE
eth0                    172.17.0.0/16

Requests for port 22 and 80 on the public IP address are redirected to ports 9022 and 9080 in the masqueraded nework:

/etc/shorewall/rules:
#
#       - 9080, 9443 is where nginx listens and requests to 80/443 are forwarded to 9080/9443
#       - 9022 is for the sftp server
ACCEPT          net     fw:86.59.12.252 tcp     80
ACCEPT          net     fw:86.59.12.252 tcp     443
ACCEPT          net     fw:86.59.12.252 tcp     9022
ACCEPT          net     fw:86.59.12.252 tcp     9080
REDIRECT        net     9080    tcp 80  -       86.59.12.252
ACCEPT          net     fw:86.59.12.252 tcp     9443
REDIRECT        net     9443    tcp 443 -       86.59.12.252
ACCEPT          net     fw:86.59.12.252         icmp
REJECT          net     fw:86.59.12.252         tcp

Replicating the distribution host setup:

• install docker and make sure docker starts automatically on reboot install docker-compose

• clone the repo https://github.com/machinekit/mk_deb_server and follow the instructions in README.md

• open up the firewall according to the above IP setup instructions

Setting up a signing key for reprepro:

The current master key is 43ddf224, see http://keyserver.ubuntu.com/pks/lookup?op=vindex&search=0x31b5958f43ddf224&fingerprint=on&exact=on

Signing happens with a subkey, which needs to be exported to /home/bin/mk_repo/keys/no_passwd_reprepro.key.

This subkey may not have a passphrase. Current subkey id is 499BE968 .

Generating a new signing subkey

Cheatsheet for key owner only (that’d be me). In case the current subkey is compromised:

Using the keyring containing 43ddf224:

• revoke the current signing subkey

• generate a new subkey of 43ddf224 for signing (DSA, 2048)

• export the updated 43ddf224 public key in the keyservers

gpg --keyserver keyserver.ubuntu.com --send-key 43ddf224 - export that new signing subkey in ascii-armor into no_passwd_reprepro.key

 — gpg --export-secret-subkeys --export-options export-reset-subkey-passwd <subkey-id> >signing.key — gpg --armor --output no_passwd_reprepro.key --export-secret-subkeys <subkey>!

• transfer key to serving host into /home/bin/mk_repo/keys/

• tell clients to update: apt-key adv --keyserver keyserver.ubuntu.com --recv 43DDF224