Custom Platform Provider

[sivchari]

I propose custom platform provider. Now, PipeCD can only manage Kubernetes/CloudRun/ECS/Lambda/Terraform.
But other than that, PipeCD can't manage. If we define specification to extend manage resources, developers can use PipeCD for many resources. wdyt ?

[khanhtc1202]

This point is excellent, and it's what we want tbh. There are several things which be listed in our roadmap already, but I didn't have time to start until today. 😓 Please join our next public community MTG for further discussion. And let's list up idea under this discussion 👍

[ffjlabo]

discussion in 2023/10/27

Note

from @sivchari

• The background of this idea
  • For the concept of PipeCD, "The One CD for All {applications, platforms, operations}" it should be better to deploy to any application platform with PipeCD
  • Can we avoid the impact of changing the Hashicorp license?
• idea
  • i.g. define an interface for executors

from @khanhtc1202

• agree with that idea!, and also have a similar idea.
• So, creating a plugin system at first should be better.
• Some things could be improved.
  • We need to think about the impact of the changes.
  • We may need to fix the data model, so the migration is needed.

memos

• Can we use https://github.com/knqyf263/go-plugin as our plugin system?

Next Action

• more discussion about it in the nearest day

[khanhtc1202]

Plugin over gRPC: https://github.com/hashicorp/go-plugin
Plugin over Wasm: https://github.com/knqyf263/go-plugin

[khanhtc1202]

About the plan, it would be better to keep the first version of pluggable piped separated from the piped source. So that we can keep develop new features for piped while refactoring the piped architect to make it pluggable.

=> Let's create: /pkg/app/pipedv1 as copy of /pkg/app/piped

[khanhtc1202]

What to concern?

1. How to communicate (between piped & plugin)
2. How to register new plugins? (if needed)
3. How to install / load plugins?
4. How about the common stages? (currently: wait / wait-approval / analysis / script-run)
5. Migration for the current data of pipeds?
   • Models which is touched by piped: piped / deployment (includes: stages)
   • Platform providers object: https://github.com/pipe-cd/pipecd/blob/master/pkg/model/piped.proto#L31-L34
   • Kind related data: application.kind / application_live_state.kind / application_planpreview_result.kind / ...

[khanhtc1202]

We will track the process via this issue 👍
#4980