Skip to content

Better SCM CI Integration Workflows

Jump to bottom
Dany Marcoux edited this page Dec 19, 2022 · 41 revisions

Workflows

The .obs/workflows.yml file can contain multiple workflows with unique names

test_build:
  steps:
    - branch_package
        source_project: OBS:Server:Unstable
        source_package: obs-server
        target_project: OBS:Server:Unstable:CI
  filters:
    event: pull_request
rebuild_master:
  steps:
    - rebuild_package
        project: home:Admin
        package: ctris
  filters:
    event: push
    branches:
      only:
        - master

Placeholder Variables

With placeholder variables, workflows are now dynamic. Whenever a webhook event comes in, OBS downloads the workflows file and parses it. This is when the placeholder variables are replaced by the data they refer to in the webhook event payload.

Here's a list of supported placeholder variables and their mapping:

  • %{SCM_ORGANIZATION_NAME}: The name of the SCM organization, like openSUSE for the GitHub repository openSUSE/open-build-service
  • %{SCM_REPOSITORY_NAME}: The name of the SCM repository, like open-build-service for the GitHub repository openSUSE/open-build-service
  • %{SCM_PR_NUMBER}: The number of the pull request
  • %{SCM_COMMIT_SHA}: The SHA of the commit which was pushed

Example of workflows with some placeholder variables:

# test_build will branch a package based on the SCM repository name from which the webhook event came from.
test_build:
  steps:
    - branch_package
        source_project: OBS:Server:Unstable
        source_package: %{SCM_REPOSITORY_NAME}
        target_project: OBS:Server:Unstable:CI
  filters:
    event: pull_request

Configuration File Location

TODO: Explain how path and URL behave for this.

Workflow Steps

Branch a Package

Equivalent of osc branch

Providing the source project OBS:Server:Unstable, the source package obs-server and the target project OBS:Server:Unstable:CI will branch¹ the package OBS:Server:Unstable/obs-server:

  • to OBS:Server:Unstable:CI:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER/obs-server for a pull request webhook event.
  • to OBS:Server:Unstable:CI/obs-server-$MY_COMMIT_SHA_OR_TAG_NAME for a push webhook event.

¹ Branching means creating a link, copying over files from the source package and creating configuring all repositories from the source project

workflow:
  steps:
    - branch_package:
        source_project: OBS:Server:Unstable
        source_package: obs-server
        target_project: OBS:Server:Unstable:CI

Link a Package to a Project

Equivalent of osc linkpac

Providing the source project OBS:Server:Unstable, the source package obs-server and the target project OBS:Server:Unstable:CI will link¹ the package OBS:Server:Unstable/obs-server:

  • to OBS:Server:Unstable:CI:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER/obs-server for a pull request webhook event.
  • to OBS:Server:Unstable:CI/obs-server-$MY_COMMIT_SHA_OR_TAG_NAME for a push webhook event.

¹ Linking means creating a link pointing to the source package

workflow:
  steps:
    - link_package:
        source_project: OBS:Server:Unstable
        source_package: obs-server
        target_project: OBS:Server:Unstable:CI

Configure Repositories/Architectures for a Project

Providing the project OBS:Server:Unstable:CI and multiple repositories with each repository having a name, a target project, a target repository and their architectures will configure the project with the provided repositories and architectures.

Please note that the provided project has to be a project which was a target project from a previous step like link_package or branch_package. The target project will be:

  • OBS:Server:Unstable:CI:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER for a pull request webhook event.
  • OBS:Server:Unstable:CI for a push webhook event.
workflow:
  steps:
    - configure_repositories:
        project: OBS:Server:Unstable:CI
        repositories:
          - name: openSUSE_Tumbleweed
            paths:
              - target_project: openSUSE:Factory
                target_repository: snapshot
              - target_project: openSUSE:Tumbleweed
                target_repository: standard
            architectures:
              - x86_64
              - i586
          - name: openSUSE_Leap_15.2
            paths:
              - target_project: openSUSE:Leap:15.2
                target_repository: standard
            architectures:
              - x86_64

Rebuild a Package

Equivalent of osc rebuild

Providing the project home:Admin and the package ctris will rebuild the package home:Admin/ctris.

This is possible only if the user of the token running the workflow has permissions to rebuild the package.

workflow:
  steps:
    - rebuild_package:
        project: home:Admin
        package: ctris

Set Flags

Providing the type build, the status enable and the project home:Admin will enable all builds:

  • for the home:Admin:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER project when the webhook event is a pull request.
  • for the home:Admin when the webhook event is a push.

Please note that the provided project has to be a project which was a target project from a previous step like link_package or branch_package. Same for the package, it has to be branched/linked in a previous step.

The type has to be one of keys from FlagHelper::TYPES.

The status is either disable or enable.

Providing multiple flags is supported as noted in the YAML below.

workflow:
  steps:
    - set_flags:
        flags:
          - type: build
            status: enable
            project: home:Admin
          - type: publish
            status: disable
            project: home:Admin

The package, repository and architecture keys are all optional. When provided, they limit the flag to a certain package, repository or architecture.

So with the YAML provided below and a pull request event, builds of the home:Admin:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER/ctris package will be disabled for the openSUSE_Tumbleweed repository and x86_64 architecture. For a push event, it's exactly the same, except for the package which is home:Admin/ctris-$MY_COMMIT_SHA_OR_TAG_NAME.

workflow:
  steps:
    - set_flags:
        flags:
          - type: build
            status: disable
            project: home:Admin
            package: ctris
            repository: openSUSE_Tumbleweed
            architecture: x86_64

Trigger Services

Equivalent of osc service

Providing the project home:Admin and the package ctris will trigger services of the package home:Admin/ctris.

This is possible only if the user of the token running the workflow has permissions to trigger services of the package.

workflow:
  steps:
    - trigger_services:
        project: home:Admin
        package: ctris

Filters in Workflows

You can make use of the Branch and Event filters which restrict workflows to run only for or ignore certain branches/events.

Filters are defined in .obs/workflows.yml. Please refer to the subsections for details on each filter.

Here are some examples with the filters:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    event: pull_request
    branches:
      only:
        - master
        - staging
workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    event: push
    branches:
      ignore:
        - staging

Event Filter

Run workflow only for a specific GitHub/GitLab/Gitea event.

For GitHub events, see https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads.

For GitLab events, see https://docs.gitlab.com/ee/user/project/integrations/webhook_events.html.

For Gitea events. see https://docs.gitea.io/en-us/webhooks/

We use SCM-independent values for events:

  • pull_request for pull/merge requests events
  • push for push events related to commits
  • tag_push for push events related to tags

Example to run workflow only for a pull/merge request event from GitHub/GitLab/Gitea:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    event: pull_request

Branches Filter

⚠️ We do not support basic regular expression in the first version of branches filter. ⚠️

Matching target branches based on their names and run workflow only for those branches. Basic regular expression is supported with ^ (starting with) and $ (ending with).

Example to run workflow only for the target branch master, all target branches starting with staging and all target branches ending with final:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    branches:
      only:
        - master
        - ^staging
        - final$

Example to run workflow for all target branches, except master, those starting with staging and those ending with final:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    branches:
      ignore:
        - master
        - ^staging
        - final$

only has precedence over ignore, so if both are defined, ignore is not considered.

Development

Subsystems

Better SCM/CI Integration

HOWTOs

Software Design Patterns

Operations

Releases

Clone this wiki locally