custom rules with multi-cluster support (Plugin API)

[calvinxu]

Just wonder how to use "custom rules with multi-cluster support (Plugin API)" feature, does it mean that we can create custom rules, what are the rules refer here?
and does this feature support require to use PluginAPI, which to be installed separately or what type of PluginAPI here?

[SOF3]

1. Annotation linker

Currently, you can link make objects declare their own parent without ownerReferences by writing the annotation kelemetry.kubewharf.io/parent-link on an object to point to its parent object with a JSON-encoded value in the ParentLink schema.

2. Custom linker plugin

Alternatively, you may use out-of-tree plugins by building your own binary with ProvideListImpl on the aggregator/linker.Linker interface. For a TLDR, it means to create a file in your workspace like this:

package main

import (
    "context"

    "github.com/kubewharf/kelemetry/cmd/kelemetry"
    "github.com/kubewharf/kelemetry/pkg/aggregator/linker"
    "github.com/kubewharf/kelemetry/pkg/manager"
    _ "github.com/kubewharf/kelemetry/pkg"
    "github.com/kubewharf/kelemetry/pkg/util"
)

type myLinker struct {
    manager.BaseComponent
}
func (*myLinker) Lookup(ctx context.Context, object util.ObjectRef) *util.ObjectRef {
    // compute new parent object reference from `object`
}

func init() {
    manager.Global.ProvideListImpl("my-linker", manager.Ptr(&myLinker{}), &manager.List[linker.Linker]{})
}

func main() {
    kelemetry.Main()
}

Then compile this binary in place of the official Kelemetry image.

3. LinkRule

There is a WIP feature in #109 (currently on hold to merge #157 first), which will allow dynamically defining link rules with a CRD.

[calvinxu]

1. so for the 1. "....by writing the annotation kelemetry.kubewharf.io/parent-link on an object to point to its parent object with a JSON-encoded value in the [ParentLink schema]" is there an example can be referenced?
2. which means this need extra effort to implement this out-of-tree plugin, and place it into the kelemetry image to make it work?
3. How to understand "allow dynamically defining link rules with a CRD" in this feature?

[SOF3]

1. This feature is not very well-documented yet, because it is a bit awkward to use — you need to ensure the annotation is present when Kelemetry processes this object the first time (i.e. on creation), which means you need either a mutating webhook or to modify the controller that creates an object. Roughly speaking it looks like this:

metadata:
  annotations:
    kelemetry.kubewharf.io/parent-link: |
      {"cluster": "the-cluster-name", "group": "apps", "version": "v1", "resource": "deployments", "namespace": "default", "name": "example"}

2. Yes, more cumbersome to implement, but most flexible since you can execute any Go code there.

3. In the current design for that feature, for example, to support linking helm-rendered objects to the helm secret, you can write a rule like this:

apiVersion: kelemetry.kubewharf.io/v1alpha1
kind: LinkRule
metadata:
  name: 
filter:
  selector:
    matchLabels:
      app.kubernetes.io/managed-by: Helm
parent:
  group: ""
  version: v1
  resource: secrets
  scope: Namespaced
  nameTemplate: '{{.metadata.annotations["meta.helm.sh/release-name"]}}'

However the feature has not been completed yet and is subject to change.

[SOF3]

By the way, with the changes in #157, it might become feasible to extract linkers to a separate process in the future using a distributed message queue (currently linkers are synchronous, and #157 makes them asynchronous, but still only supports a local message queue), then the custom linkers may be deployed like add-on controllers instead of replacing the kelemetry image directly.