Updatecli: Use separated pipelines + organization scanning for all updatecli processes in jenkins-infra

[dduportal]

• No more GitHub actions for updatecli (less configuration spread)
  • For instance Update updatecli github action used jenkins-infra#2059
  • Full control of the execution environment of updatecli
• Updatecli pipeline triggers can be controlled separately from repository own pipelines
  • Simplifies repository own pipelines (less conditions)
  • Allow to fine tune updatecli frequency: weekly, daily, etc.
  • repository own pipelines would be completely separated from the "chore" tasks of updatecli
• Centralized credentials in infra.ci.jenkins.io (which is private)

Convention Proposal:

• Use a Jenkinsfile_updatecli a pipeline marker
• Define a new GH organization scanning on infra.ci.jenkins.io that will check ALL of jenkins-infra and will check for these updatecli processes
• Write down a documentation of this in jenkins-infra/documentation

Current progress: #2778 (comment)

[dduportal]

Blocked by #2834 (because of involved credentials)

[dduportal]

Status:

• PoC nice on kubernetes-management

Next step:

• As it's per-repo basis, we have to implement GH org. scanning (wip on helm chart)
• Slightly minor issue: the 2 jobs have the same GH check status. How to change that?

[lemeurherve]

While they were opened, I updated my PRs to use https://github.com/apps/jenkins-infra-updatecli instead of https://github.com/jenkins-infra-bot in updatecli values. (No more co-authored commits with both updatecli GitHub app and jenkins-infra-bot GitHub bot account)

[dduportal]

Update:

• https://github.com/jenkins-infra/azure-net done
• For all terraform, we are having issues regarding the pipeline con trigger not updated on azure or azure net. Worth checking the pipeline library for terraform: it was recommend NOT to set up pipeline triggers in the library (but per Jenkins file instead)
• All GHA > Jenkins changes won't be done as it requires more discussions (mainly: we do not want hiding the build logs to contributors on most of these jobs, hence the use of GHA)

Next steps: all terraform jobs

[dduportal]

Applied jenkins-infra/digitalocean#204

[dduportal]

Applied jenkins-infra/cloudflare#36

[dduportal]

Another one: jenkins-infra/shared-tools#159

[dduportal]

2 new ones:

• chore: separate updatecli to its own pipeline aws#558
• chore: separate updatecli to its own pipeline fastly#113

[dduportal]

This issue now requires the Docker Pipeline Library to be updated to remove the updatecli part from it.

[lemeurherve]

FWIW, I've updated the status in #2778 (comment)

FYI, there are 11 remaining active repositories using parallelDockerUpdatecli shared pipeline library function: https://github.com/search?q=org%3Ajenkins-infra%20parallelDockerUpdatecli&type=code

2 of them don't have a PR yet. (https://github.com/jenkins-infra/docker-geoipupdate & https://github.com/jenkins-infra/docker-keycloak-theme)

[dduportal]

FWIW, I've updated the status in #2778 (comment)

FYI, there are 11 remaining active repositories using parallelDockerUpdatecli shared pipeline library function: https://github.com/search?q=org%3Ajenkins-infra%20parallelDockerUpdatecli&type=code

2 of them don't have a PR yet. (https://github.com/jenkins-infra/docker-geoipupdate & https://github.com/jenkins-infra/docker-keycloak-theme)

Thanks for the reminder. Please note that the PRs related to parallelDockerUpdatecli can be closed as we won't merge them given the amount of requirements: it's already been a few months and realistically a lot more months will pass before we can work on the pipeline library. (as such no need to open PR for the 2 other repos).