-
Notifications
You must be signed in to change notification settings - Fork 216
Add instructions for running a openQA/tools dev environment #6590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
perlpunk
wants to merge
3
commits into
os-autoinst:master
Choose a base branch
from
perlpunk:devcontainer
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+233
−0
Draft
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -182,6 +182,10 @@ tools:: Development tools | |
|
||
[[development-setup]] | ||
== Development setup | ||
|
||
For the lazy, check out the <<Contributing.asciidoc#quick-container-setup,Quick | ||
container setup>> with instructions you can mostly copy & paste. | ||
|
||
For developing openQA and os-autoinst itself it makes sense to checkout the | ||
<<Contributing.asciidoc#repo-urls,Git repositories>> and either execute | ||
existing tests or start the daemons manually. | ||
|
@@ -431,6 +435,235 @@ Also find more details in | |
look at the | ||
https://github.com/Martchus/openQA-helper[openQA-helper repository]. | ||
|
||
[[quick-container-setup]] | ||
=== Quick container setup | ||
|
||
==== Requirements | ||
|
||
In these examples we use the following tools which are all in openSUSE Leap: | ||
|
||
* https://podman.io/[`podman`] | ||
* https://en.opensuse.org/Distrobox[`distrobox`] | ||
|
||
You can use similar tools like `docker`, `toolbox` etc. to achieve the same. | ||
|
||
==== Intro | ||
|
||
These instructions are ready to use without any changes. You only might want to | ||
change the `OPENQA_BASEDIR` location. | ||
|
||
You will run two containers: | ||
|
||
* A separate postgres container (so you don't have to deal with the database | ||
when updating your dev container) | ||
* The openQA environment in a openSUSE Leap or Tumbleweed container, which | ||
both can use the same postgres container. | ||
In this example we will be settting up and using the Leap container. | ||
|
||
`distrobox`, for example, can be used, to take care of network, home directory, | ||
permissions, and graphical environment. | ||
|
||
Some common settings we are using here: | ||
|
||
* `OPENQA_BASEDIR=$HOME/openqadev.leap` | ||
* Database name: `openqa-local` | ||
* Database user: `geekotest` | ||
* Database container name: `postgres-openqa` | ||
|
||
==== Environment Variables | ||
|
||
Create a file that you can `source` whenever you want to work in this | ||
environment: | ||
|
||
File: `~/localopenqa.leap.sh`: | ||
[source,sh] | ||
---- | ||
#!/bin/bash | ||
|
||
# The directory where openQA operates in and which creates everything: | ||
# * git repos | ||
# * openQA assets, pool directory, testresults | ||
# * configuration | ||
|
||
export OPENQA_BASEDIR=$HOME/openqadev.leap | ||
|
||
# The rest of the variables can likely stay like this | ||
|
||
# If you want to import database dumps from elsewhere | ||
#export SQL_DUMP_DIR=$HOME/sqldumps | ||
|
||
export OPENQA_CONFIG=$OPENQA_BASEDIR/config | ||
export OPENQA_REPOS="$OPENQA_BASEDIR/git" | ||
PATH=$OPENQA_REPOS/openQA/scripts:$PATH | ||
|
||
# ----------- CPAN ------------ | ||
# Whenever necessary, you can install additional modules from CPAN like: | ||
# cpanm -l $OPENQA_BASEDIR/perl5 Some::[email protected] | ||
export PERL5LIB=$OPENQA_BASEDIR/perl5/lib/perl5 | ||
PATH=$OPENQA_BASEDIR/perl5/bin:$PATH | ||
|
||
export OPENQA_KEY=1234567890ABCDEF | ||
export OPENQA_SECRET=1234567890ABCDEF | ||
---- | ||
|
||
==== Postgres Container | ||
|
||
To be able to keep the database when stopping the container, we create a | ||
named volume. | ||
We are using a Debian postgres container here, but it doesn't matter what | ||
you use. | ||
[source,sh] | ||
---- | ||
### host ### | ||
podman volume create mypostgres | ||
podman run -it --rm --name postgres-openqa \ | ||
-v mypostgres:/var/lib/postgresql/data \ | ||
-p 5432:5432 \ | ||
-e POSTGRES_PASSWORD="x" \ | ||
-d \ | ||
docker.io/library/postgres:17 | ||
|
||
# Optionally add this to mount a directory for sql dumps to import | ||
# -v "$SQL_DUMP_DIR:/sqldumps" \ | ||
---- | ||
|
||
For the next step you need to execute commands in the container: | ||
|
||
[source,sh] | ||
---- | ||
# Create the role and database | ||
podman exec -it postgres-openqa su - postgres | ||
|
||
createuser --no-createdb --pwprompt geekotest | ||
# enter password ('x' in this example) | ||
createdb -O geekotest openqa-local | ||
---- | ||
|
||
==== Distrobox | ||
|
||
You can use a leap container and install the requirements, but you can also | ||
build your own container with the requirements already installed. | ||
|
||
[source,sh] | ||
---- | ||
### host ### | ||
distrobox create -i registry.opensuse.org/opensuse/leap:15.6 -n box-openqadev | ||
distrobox enter box-openqadev | ||
|
||
### distrobox ### | ||
|
||
# Leap 15.6 | ||
zypper addrepo -G -p 95 https://download.opensuse.org/repositories/devel:/openQA/15.6 devel-openqa \ | ||
&& zypper addrepo -G -p 95 https://download.opensuse.org/repositories/devel:/openQA:/Leap:/15.6/15.6/ devel-openqa-leap | ||
|
||
# Tumbleweed | ||
#zypper addrepo -G p 95 https://download.opensuse.org/repositories/devel:/openQA/openSUSE_Tumbleweed devel-openqa \ | ||
# && zypper addrepo -G -p 95 https://download.opensuse.org/repositories/SUSE:/CA/openSUSE_Tumbleweed/SUSE:CA.repo | ||
|
||
zypper -n ref && zypper -n install ca-certificates-suse \ | ||
os-autoinst-devel openQA-devel \ | ||
os-autoinst-scripts-deps \ | ||
os-autoinst-scripts-deps-devel \ | ||
openssh-clients \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. openssh-clients is necessary to do Should we add it to os-autoinst and openQA requirements? |
||
crudini | ||
# plus your favourite dev tools, e.g. tmux | ||
---- | ||
Note: https://github.com/pixelb/crudini[`crudini`] is a tool for manipulating | ||
values in ini files. | ||
|
||
Time to grab a hot beverage... | ||
|
||
==== Git | ||
|
||
[source,sh] | ||
---- | ||
### distrobox ### | ||
# Remember to source this file | ||
source ~/localopenqa.leap.sh | ||
mkdir $OPENQA_BASEDIR | ||
mkdir $OPENQA_REPOS | ||
mkdir $OPENQA_BASEDIR/openqa $OPENQA_CONFIG $OPENQA_BASEDIR/perl5 $OPENQA_BASEDIR/openqa/db | ||
|
||
git clone [email protected]:os-autoinst/openQA $OPENQA_REPOS/openQA | ||
git clone [email protected]:os-autoinst/os-autoinst $OPENQA_REPOS/os-autoinst | ||
git clone [email protected]:os-autoinst/scripts $OPENQA_REPOS/scripts | ||
---- | ||
|
||
==== Config Files | ||
|
||
[source,sh] | ||
---- | ||
### distrobox ### | ||
echo <<< EOM | ||
[localhost] | ||
key = 1234567890ABCDEF | ||
secret = 1234567890ABCDEF | ||
EOM >> $OPENQA_CONFIG/client.conf | ||
|
||
echo <<< EOM | ||
[production] | ||
dsn = dbi:Pg:dbname=openqa-local;host=127.0.0.1;port=5432 | ||
user = geekotest | ||
password = x | ||
EOM >> $OPENQA_CONFIG/database.ini | ||
|
||
cp $OPENQA_REPOS/openQA/etc/openqa/workers.ini $OPENQA_CONFIG/workers.ini | ||
crudini --set $OPENQA_CONFIG/workers.ini global HOST http://localhost:9526 | ||
crudini --set $OPENQA_CONFIG/workers.ini global WORKER_HOSTNAME 127.0.0.1 | ||
|
||
cp $OPENQA_REPOS/openQA/etc/openqa/openqa.ini $OPENQA_CONFIG/openqa.ini | ||
crudini --set $OPENQA_CONFIG/openqa.ini auth method Fake | ||
---- | ||
|
||
|
||
==== Build openQA | ||
|
||
[source,sh] | ||
---- | ||
### distrobox ### | ||
# Build steps | ||
cd $OPENQA_REPOS/os-autoinst | ||
make | ||
cd $OPENQA_REPOS/openQA | ||
make node_modules | ||
|
||
# Optionally check if you can access the database | ||
#psql -U geekotest -h 127.0.0.1 -p 5432 openqa-local | ||
|
||
# Initialize the database schema. This is using the database.ini you | ||
# created above. | ||
$OPENQA_REPOS/openQA/script/initdb --init_database | ||
---- | ||
|
||
==== Run openQA | ||
|
||
Ready! | ||
|
||
For playing with the Webui, you just need to run this one daemon. | ||
For running openQA jobs, you need all. | ||
|
||
[source,sh] | ||
---- | ||
# WebUI | ||
$OPENQA_REPOS/openQA/script/openqa-webui-daemon | ||
|
||
# Worker | ||
worker --isotovideo "$OPENQA_REPOS/os-autoinst/isotovideo" --instance 1 --verbose --apikey $OPENQA_KEY --apisecret $OPENQA_SECRET | ||
|
||
# Websocket | ||
openqa-websockets daemon | ||
|
||
# Scheduler | ||
openqa-scheduler daemon | ||
|
||
# Gru (Minion) | ||
openqa gru run | ||
|
||
# Lievehandler | ||
openqa-livehandler daemon | ||
---- | ||
|
||
|
||
[[dependency-handling]] | ||
== Handling of dependencies | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the very first line of the
Development setup
and it would be more enlightening to describe the options or the differences from other sections. Something likeIn order to contribute, your development environment needs the following steps [blabla].. In order to setup a containerized env follow these instructions....[]