Skip to content

Add instructions for running a openQA/tools dev environment #6590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
233 changes: 233 additions & 0 deletions docs/Contributing.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,10 @@ tools:: Development tools

[[development-setup]]
== Development setup

For the lazy, check out the <<Contributing.asciidoc#quick-container-setup,Quick
container setup>> with instructions you can mostly copy & paste.
Comment on lines +186 to +187
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the very first line of the Development setup and it would be more enlightening to describe the options or the differences from other sections. Something like

In order to contribute, your development environment needs the following steps [blabla].. In order to setup a containerized env follow these instructions....[]


For developing openQA and os-autoinst itself it makes sense to checkout the
<<Contributing.asciidoc#repo-urls,Git repositories>> and either execute
existing tests or start the daemons manually.
Expand Down Expand Up @@ -431,6 +435,235 @@ Also find more details in
look at the
https://github.com/Martchus/openQA-helper[openQA-helper repository].

[[quick-container-setup]]
=== Quick container setup

==== Requirements

In these examples we use the following tools which are all in openSUSE Leap:

* https://podman.io/[`podman`]
* https://en.opensuse.org/Distrobox[`distrobox`]

You can use similar tools like `docker`, `toolbox` etc. to achieve the same.

==== Intro

These instructions are ready to use without any changes. You only might want to
change the `OPENQA_BASEDIR` location.

You will run two containers:

* A separate postgres container (so you don't have to deal with the database
when updating your dev container)
* The openQA environment in a openSUSE Leap or Tumbleweed container, which
both can use the same postgres container.
In this example we will be settting up and using the Leap container.

`distrobox`, for example, can be used, to take care of network, home directory,
permissions, and graphical environment.

Some common settings we are using here:

* `OPENQA_BASEDIR=$HOME/openqadev.leap`
* Database name: `openqa-local`
* Database user: `geekotest`
* Database container name: `postgres-openqa`

==== Environment Variables

Create a file that you can `source` whenever you want to work in this
environment:

File: `~/localopenqa.leap.sh`:
[source,sh]
----
#!/bin/bash

# The directory where openQA operates in and which creates everything:
# * git repos
# * openQA assets, pool directory, testresults
# * configuration

export OPENQA_BASEDIR=$HOME/openqadev.leap

# The rest of the variables can likely stay like this

# If you want to import database dumps from elsewhere
#export SQL_DUMP_DIR=$HOME/sqldumps

export OPENQA_CONFIG=$OPENQA_BASEDIR/config
export OPENQA_REPOS="$OPENQA_BASEDIR/git"
PATH=$OPENQA_REPOS/openQA/scripts:$PATH

# ----------- CPAN ------------
# Whenever necessary, you can install additional modules from CPAN like:
# cpanm -l $OPENQA_BASEDIR/perl5 Some::[email protected]
export PERL5LIB=$OPENQA_BASEDIR/perl5/lib/perl5
PATH=$OPENQA_BASEDIR/perl5/bin:$PATH

export OPENQA_KEY=1234567890ABCDEF
export OPENQA_SECRET=1234567890ABCDEF
----

==== Postgres Container

To be able to keep the database when stopping the container, we create a
named volume.
We are using a Debian postgres container here, but it doesn't matter what
you use.
[source,sh]
----
### host ###
podman volume create mypostgres
podman run -it --rm --name postgres-openqa \
-v mypostgres:/var/lib/postgresql/data \
-p 5432:5432 \
-e POSTGRES_PASSWORD="x" \
-d \
docker.io/library/postgres:17

# Optionally add this to mount a directory for sql dumps to import
# -v "$SQL_DUMP_DIR:/sqldumps" \
----

For the next step you need to execute commands in the container:

[source,sh]
----
# Create the role and database
podman exec -it postgres-openqa su - postgres

createuser --no-createdb --pwprompt geekotest
# enter password ('x' in this example)
createdb -O geekotest openqa-local
----

==== Distrobox

You can use a leap container and install the requirements, but you can also
build your own container with the requirements already installed.

[source,sh]
----
### host ###
distrobox create -i registry.opensuse.org/opensuse/leap:15.6 -n box-openqadev
distrobox enter box-openqadev

### distrobox ###

# Leap 15.6
zypper addrepo -G -p 95 https://download.opensuse.org/repositories/devel:/openQA/15.6 devel-openqa \
&& zypper addrepo -G -p 95 https://download.opensuse.org/repositories/devel:/openQA:/Leap:/15.6/15.6/ devel-openqa-leap

# Tumbleweed
#zypper addrepo -G p 95 https://download.opensuse.org/repositories/devel:/openQA/openSUSE_Tumbleweed devel-openqa \
# && zypper addrepo -G -p 95 https://download.opensuse.org/repositories/SUSE:/CA/openSUSE_Tumbleweed/SUSE:CA.repo

zypper -n ref && zypper -n install ca-certificates-suse \
os-autoinst-devel openQA-devel \
os-autoinst-scripts-deps \
os-autoinst-scripts-deps-devel \
openssh-clients \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openssh-clients is necessary to do git clone git@...

Should we add it to os-autoinst and openQA requirements?

crudini
# plus your favourite dev tools, e.g. tmux
----
Note: https://github.com/pixelb/crudini[`crudini`] is a tool for manipulating
values in ini files.

Time to grab a hot beverage...

==== Git

[source,sh]
----
### distrobox ###
# Remember to source this file
source ~/localopenqa.leap.sh
mkdir $OPENQA_BASEDIR
mkdir $OPENQA_REPOS
mkdir $OPENQA_BASEDIR/openqa $OPENQA_CONFIG $OPENQA_BASEDIR/perl5 $OPENQA_BASEDIR/openqa/db

git clone [email protected]:os-autoinst/openQA $OPENQA_REPOS/openQA
git clone [email protected]:os-autoinst/os-autoinst $OPENQA_REPOS/os-autoinst
git clone [email protected]:os-autoinst/scripts $OPENQA_REPOS/scripts
----

==== Config Files

[source,sh]
----
### distrobox ###
echo <<< EOM
[localhost]
key = 1234567890ABCDEF
secret = 1234567890ABCDEF
EOM >> $OPENQA_CONFIG/client.conf

echo <<< EOM
[production]
dsn = dbi:Pg:dbname=openqa-local;host=127.0.0.1;port=5432
user = geekotest
password = x
EOM >> $OPENQA_CONFIG/database.ini

cp $OPENQA_REPOS/openQA/etc/openqa/workers.ini $OPENQA_CONFIG/workers.ini
crudini --set $OPENQA_CONFIG/workers.ini global HOST http://localhost:9526
crudini --set $OPENQA_CONFIG/workers.ini global WORKER_HOSTNAME 127.0.0.1

cp $OPENQA_REPOS/openQA/etc/openqa/openqa.ini $OPENQA_CONFIG/openqa.ini
crudini --set $OPENQA_CONFIG/openqa.ini auth method Fake
----


==== Build openQA

[source,sh]
----
### distrobox ###
# Build steps
cd $OPENQA_REPOS/os-autoinst
make
cd $OPENQA_REPOS/openQA
make node_modules

# Optionally check if you can access the database
#psql -U geekotest -h 127.0.0.1 -p 5432 openqa-local

# Initialize the database schema. This is using the database.ini you
# created above.
$OPENQA_REPOS/openQA/script/initdb --init_database
----

==== Run openQA

Ready!

For playing with the Webui, you just need to run this one daemon.
For running openQA jobs, you need all.

[source,sh]
----
# WebUI
$OPENQA_REPOS/openQA/script/openqa-webui-daemon

# Worker
worker --isotovideo "$OPENQA_REPOS/os-autoinst/isotovideo" --instance 1 --verbose --apikey $OPENQA_KEY --apisecret $OPENQA_SECRET

# Websocket
openqa-websockets daemon

# Scheduler
openqa-scheduler daemon

# Gru (Minion)
openqa gru run

# Lievehandler
openqa-livehandler daemon
----


[[dependency-handling]]
== Handling of dependencies

Expand Down