Skip to content

Conversation

@carlesarnal
Copy link
Member

@carlesarnal carlesarnal commented Dec 4, 2025

Summary

This PR simplifies and optimizes the Apicurio Registry CI/CD pipeline by restructuring the verification workflows into a modular, parallel architecture.

Changes

  • Modular workflow structure: Replaced monolithic workflows with verify.yaml orchestrating specialized sub-workflows (verify-build.yaml, verify-unit-tests.yaml, verify-integration-tests.yaml, etc.)
  • Parallel test execution: Integration tests now run as 11 parallel matrix jobs (3 storages × 3 profiles + 2 extras) instead of sequential execution
  • Reusable workflows: Added reusable-build.yaml, reusable-docker-build.yaml, and reusable-notify-slack.yaml to eliminate code duplication
  • Optimized builds: Unit tests run in parallel using Surefire forks against pre-built artifacts
  • Updated action versions: All GitHub Actions updated to v4/v5

Test plan

  • Verify all workflow phases execute correctly
  • Confirm integration test matrix runs in parallel
  • Validate Docker image publishing on main branch pushes
  • Check Slack notifications trigger appropriately

@carlesarnal carlesarnal force-pushed the simplify-apicurio-testing branch from fe3d3a4 to 0499990 Compare December 5, 2025 07:50
@carlesarnal carlesarnal force-pushed the simplify-apicurio-testing branch from 2842f75 to 1332240 Compare December 5, 2025 08:53
@carlesarnal carlesarnal force-pushed the simplify-apicurio-testing branch from b8682d7 to ec58712 Compare December 9, 2025 09:06
@carlesarnal carlesarnal force-pushed the simplify-apicurio-testing branch from 7e230e3 to f47173f Compare December 9, 2025 10:58
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not need to be committed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but if you read it, you'll get a much better idea of what's going on. I'm going to update the PR description with this.

@jsenko
Copy link
Member

jsenko commented Dec 9, 2025

This is pretty big. I trust you if this works, is there anything in particular that you want opinion on?

@carlesarnal
Copy link
Member Author

This is pretty big. I trust you if this works, is there anything in particular that you want opinion on?

About the approach itself. I'm not asking for a full review of the thing :)

The TLDR is essentially the following:

  • Lint and validate. db lint, docs verification.
  • Build: build java backend, ui, publish temporary docker images and make maven artifacts available to the rest of the jobs
  • Unit tests: the obvious, mvn verify against the already build artifacts. No re-building of the maven modules.
  • Integration tests: the same workflows as before, running in a matrix, so we can run them in parallel.
  • Extra tests: tests ui, legacy v2, typescript sdk integration, examples
  • sdk verification: python sdk, go sdk
  • native build: building native executable and running integration tests

EricWittmann
EricWittmann previously approved these changes Dec 9, 2025
Copy link
Member

@EricWittmann EricWittmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve in principal. :) Happy to have it merged and we can see how it goes and fix/iterate if needed.

@jsenko
Copy link
Member

jsenko commented Dec 9, 2025

I'm a big fan of reusable workflows and matrix execution.... I want to do that in apicurio-testing as well. So thumbs up from me! We definitely accumulated some unnecessary complexity over time.

Additional note: I don't see any changes to the operator workflows, correct? Because they aren't going to run in this PR unless there's a change in the operator dir.

@carlesarnal
Copy link
Member Author

I'm a big fan of reusable workflows and matrix execution.... I want to do that in apicurio-testing as well. So thumbs up from me! We definitely accumulated some unnecessary complexity over time.

Additional note: I don't see any changes to the operator workflows, correct? Because they aren't going to run in this PR unless there's a change in the operator dir.

Correct, this is simplifying the workflows and remove some unused ones, like the RHBOQ one. Regarding the operators, the only change is to re-use the slack notification one, that was replicated across the different ones, and one it's simplifies into a single, parametrized one.

@carlesarnal
Copy link
Member Author

All the workflows are green. I'm merging this and we can continue iterating on it. This is the first round to improve our processes. I'm also trying to improve the speed, ~30min is way too much.

@carlesarnal carlesarnal merged commit 2faf48d into Apicurio:main Dec 10, 2025
31 checks passed
@carlesarnal carlesarnal added this to the 3.1.5 milestone Dec 10, 2025
carlesarnal added a commit to carlesarnal/apicurio-registry that referenced this pull request Dec 12, 2025
* Run app tests in parallel using surefire forks

* Simplify workflows

* Re-use workflows

* Move reusable workflows to top directory

* Split verification process into multiple processes

* Build application first and run tests in parallel

* Do not build or run in parallel

* Split native workflow

Implement authentication for the mcp server
carlesarnal added a commit to carlesarnal/apicurio-registry that referenced this pull request Dec 12, 2025
* Run app tests in parallel using surefire forks

* Simplify workflows

* Re-use workflows

* Move reusable workflows to top directory

* Split verification process into multiple processes

* Build application first and run tests in parallel

* Do not build or run in parallel

* Split native workflow

Implement authentication for the mcp server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants