Development Environment Overview

This page is a detailed description of our docker based development environment. For and introduction how to use it check out our contribution guide.

Images & Services

There are many different docker images and docker-compose services that our development environment is based upon.

base

This image is the basis of all the other images. It adds our repositories, installs shared packages and sets up our user on top of our operating system (openSUSE Leap).

backend

This image installs all packages and sets up the configuration needed to run the OBS backend on top of the base image. It is used to run the backend and worker services with docker-compose. This image needs to be changed if the backend has as new dependency or configuration.

memcached

This image installs all the packages needed to run memached on top the base image and is used to run the cache service with docker-compose. This image only needs to be changed if want to run memcached differently.

mariadb

This image installs the mariadb package on top the base image and is used to run the db service with docker-compose. This image only needs to be changed if want to run mariadb differently.

frontend-base

This image installs all packages needed for our ruby on rails app on top of the base image. Additionally it serves as a kind of cache for the gem bundle. This image gets automatically rebuild on every commit by the docker hub so the gem bundle cache is up to date.

frontend

The frontend image is a thin layer on top of the frontend-base image that makes sure different users can run the ruby on rails app with their own user id. Mostly, so it does not generate log files, assets etc. in the git checkout with some strange user id. It is used to run the frontend service with docker-compose. This image needs to be rebuild every time the gem bundle` changes.

This image is different to all other images. It is build by every developer, so it can pick up their user id, and is not on the docker hub.

ahm (Application Health Monitoring)

The ahm stack is a collection of upstream docker images needed to run the full Application Health Monitoring (AHM) stack. The images involved (influxdb, rabbitmq, telegraf and grafana) as well as their configuration can be found in docker-compose.ahm.yml.

To set up the ahm stack run rake docker:ahm:prepare. This will fetch all images and pre-configure them (rake dev:ahm:configure).

The full stack can be started with docker-compose -f docker-compose.ahm.yml -f docker-compose.yml up. Please note when you want to attach to any of the services you always need to prepend -f docker-compose.yml to the command.

You can access the following services:

• Grafana: http://localhost:8000 (admin/admin)
• RabbitMQ: http://localhost:15672 (guest/guest)

ha (HAProxy setup)

The ha setup uses HAProxy to load balance the requests between two frontend hosts. The docker-compose.ha.yml file defines the proxy container (upstream HAProxy container image) and a second frontend (frontend_node_2). The cache, db, backend and worker containers are shared between the frontends.

It can bestarted with: docker-compose -f docker-compose.yml -f docker-compose.ha.yml up. The HAProxy brings also a nice and simple statistics page which can be accessed via: http://localhost:32700/ (Admin/opensuse).

Services (docker-compose)

Available services in our development environment are:

• db
• cache
• backend
• worker
• frontend

Where it makes sense they expose their service also to your host machine so you can inspect the services with the standard tools. For instances you can connect to the db service via the standard SQL tools on localhost:3306.

The services also have their dependencies defined so they will bring up everything they need (for instance the worker will start a backend, if it isn't running).

Tips & Tricks

A collection of ways you can achieve what you want to do with our development environment. This will not and can not replace docker and docker-compose documentation. You are strongly encouraged get familiar with those tools!

Services

Start/Stop/Restart all services

• docker-compose up
• docker-compose stop
• docker-compose restart

Start/Stop/Restart a single service

• docker-compose up cache
• docker-compose stop backend
• docker-compose restart db

Check running services

docker-compose ps

Attach to a running service

docker-compose exec frontend /bin/bash -l

Running commands in services

You can run any command in running services by calling docker-compose exec. This command needs two parameters, the service you want to use and the command you want to execute.

For example, to run a single test call

docker-compose exec frontend rspec spec/mixins/statistics_calculations_spec.rb

Debugging an rspec spec with pry

1. docker-compose up
2. Add binding.pry to some spec
3. docker-compose exec frontend rspec spec/mixins/statistics_calculations_spec.rb

Debugging the rails app with pry

Debugging the running rails application will start the pry session in the rails output so you need to start the rails server on it's own.

1. Add binding.pry to some code path
2. docker-compose up -d worker backend cache db
3. docker-compose run --rm --service-ports frontend rails server
4. Access your code path and pry will run

Run a single test file in the old api test suite

You can run the full test suite like we run it in CI with:

rake docker:test:minitest

To run a single test, best option currently is to edit contric/start_minitest file and change the command bundle exec rake test:api to bundle exec rake test:api TEST=path/to/test.rb.

(Re)create your database

docker-compose run --rm frontend rake db:drop dev:bootstrap

Containers

Remove all containers

1. docker-compose stop
2. docker-compose rm

Remove a single container

1. docker-compose stop frontend
2. docker-compose rm frontend

Images

Remove all images

docker image prune

Remove a single image

docker image rm openbuildservice/frontend-base

Change a static image

If you need to change one of the static images/Dockerfiles you can rebuild your local images with

rake docker:maintainer:rebuild

To publish the containers to the docker hub (you need access to the openbuildservice organization). Note that new docker images are automatically built whenever master changes. This happens via the GitHub docker service.

If needed, you can manually publish the built docker files with

rake docker:maintainer:publish

Change the frontend image

If you need to change the frontend Dockerfile you can rebuild your local image with

docker-compose build frontend

in your pull request that changed the Dockerfile let the other developers (@openSUSE/open-build-service) know that they need to rebuild their frontend container...

Rake tasks

There are a couple of rake tasks that you can run from the root of your git checkout. They are mainly used for our CI cycle but of course you can run them also on your machine.

Run the frontend test suite (only rspec so far)

rake docker:test:rspec

Run the backend test suite

rake docker:test:backend

Lint the code-base

rake docker:test:lint

Run the api test suite, all tests

rake docker:test:minitest

Setup testdata for development

Please note that this needs to get executed inside the frontend container.

rake dev:development_testdata:create

This will create some projects, a submit request, an interconnect to build.o.o and a maintenance project.

Reset everything and start from scratch 💣

Sometimes you just have to go back to where you have started....

1. docker-compose stop
2. docker-compose rm
3. docker image prune -a
4. rake docker:build
5. docker-compose up

Troubleshooting

Stale Rails server pid file

The frontend fails to start and you see a message like

frontend_1  | 09:25:14 web.1     | A server is already running. Check /obs/src/api/tmp/pids/server.pid.
...
openbuildservice_frontend_1 exited with code 1

That can happen if a docker instance forcefully get's shut down and thus doesn't clean up properly. To solve this delete the pid file with

rm  tmp/pids/server.pid

The bundle is outdated

The frontend fails to start and you see a message like

frontend_1  | 10:53:17 web.1     | /usr/lib64/ruby/gems/2.4.0/gems/bundler-1.13.6/lib/bundler/definition.rb:179:in
`rescue in specs': Your bundle is locked to mail (2.7.0), but that version could not be found in any of the sources
listed in your Gemfile. If you haven't changed sources, that means the author of mail (2.7.0) has removed it. You'll 
need to update your bundle to a different version of mail (2.7.0) that hasn't been removed in order to install. 
(Bundler::GemNotFound)
...
openbuildservice_frontend_1 exited with code 1

That can happen if you or someone else changed our bundle by updating a gem version or introducing a new gem. You have to bundle install again in your frontend container. You can do this simply by

docker-compose up --build