icyCI

icyCI has one purpose:

• Pull and verify source from a Git repository
• Run a build/test/whatever script
• Push script results to a Git repository
• Poll for new source changes

It differs from other CI utilities in a few ways:

What?	How?
Lightweight	Single Go binary with a Git core tools dependency.
Secure	Only proceeds if branch GPG/ssh signature can be verified.
Trustless	Doesn't need push access to source repositories. Rootless.
Vendor Neutral	Only relies on standard Git features and protocols.
Fast	Asynchronous, event based architecture.
Informative	Results published as git-notes, viewable from git log.
Simple	Set and forget. Polls and tests new changes automatically.
Distributed	Can cooperatively run on a single or multiple hosts.

Usage

The example below demonstrates how icyCI can be used to test the Linux kernel.

Import GPG keys used for source repository signing. Only Linus' and Greg's keys are imported below:

gpg2 --locate-keys [email protected] [email protected]

In case you haven't already, ensure that the user running icyCI has a Git user.name configured:

git config --global user.name "icyCI"

Create a new Git repository to store the test results. This example uses a local directory, but a remote repository could also be used.

git init --bare ~/icyci-linux-results

Write a test script. The example below only performs a kernel build:

echo '#!/bin/bash
      make defconfig && make -j4' > ~/build-linux.sh
chmod 755 ~/build-linux.sh

Start icyCI, pointing it at a branch of the stable kernel repository, the test script, and the results repository:

icyci -source-repo git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git \
	-source-branch linux-5.3.y \
	-test-script ~/build-linux.sh \
	-results-repo ~/icyci-linux-results

[Wait for icyCI to push results, after which it'll poll for source changes]

To view the test script results, you can either do a fresh clone of the results-repo:

git clone ~/icyci-linux-results ~/linux-results
cd ~/linux-results
git fetch origin "refs/notes/*:refs/notes/*"

...or alternatively add the results-repo as a new remote to an existing clone of the source-repo:

# The linux directory is an existing clone of
# git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux
git remote add icyci-results ~/icyci-linux-results
# git doesn't fetch notes by default, so configure it to do so...
git config --add remote.icyci-results.fetch "refs/notes/*:refs/notes/*"
git fetch icyci-results

The icyCI test results can be viewed alongside the regular git log output with:

git log --show-notes="*"

Extra build / test artifacts can be pushed to the results-repo as git notes by writing a file to the ICYCI_NOTES_DIR directory. E.g.

echo 'cp vmlinux ${ICYCI_NOTES_DIR}/built_kernel' >> ~/build-linux.sh

icyCI can be run as a systemd service. For details, see docs/systemd-usage.md.

The ability to use multiple icyCI instances to target a single repository is documented further in docs/multi-instance.md.

Architecture

icyCI runs as a relatively simple state machine:

               clone source branch
                           ↓
     ↗---→---→ check GPG signature on branch tip  → can't verify ---↘
     |                     ↓                                        |
     |         push lock to results repository    → already locked →|
     |                     ↓                                        |
     ↑         run test script                                      ↓
     |                     ↓                                        |
     |         add captured script output via git-notes             |
     |                     ↓                                        |
     ↑         push notes and source results repository             |
    new                    ↓                                        |
  commits      cleanup source repository                            |
     |                     ↓                                        |
     ↖---←---← fetch commits in source branch ←---←---←---←----←----↙
                           ↺
                        no new
                        commits

Future

• Improve documentation
• Support multiple branches within one instance
• push results as static website, in addition to git-notes
• Periodically push progress while testing
• Your feature; please raise requests via the issue tracker

License

icyCI is Free Software, published under the GNU Affero General Public License v3.0. This license applies to icyCI source itself; it does not affect programs tested by icyCI.