Cannot modify network routing peers to custom peers group

[zeinoux]

Description:

We've installed Kubernetes operator on a Managed EKS cluster using official helm from docs, but we encountered a routing bottleneck when installing nb-operator on another EKS cluster.

Upon investigation, we discovered that the network resources output in the routing peer section matched the output of the default peer name.

After we tried customizing, to try deleting the default "kubernetes" peer and creating one that we wanted, the peer routing would reset back to the default settings when we tried restarting the nb-operator deployment, or tried a helm upgrade.

I have also tried patching the NB Group CRD, with the command:

kubectl patch nbgroup router -n namespace --type merge -p '{"spec":{"name":"my-custom-group"}}'

However, this cannot be done because spec.name is immutable.

The NBGroup "router" is invalid: spec.name: Invalid value: "string": Value is immutable

Is there a solution to this issue? Or is there a configuration I'm missing for kubernetes-operator with multiple k8s clusters?

This is our custom helm values.yml :

# by default the managementURL points to the NetBird cloud service: https://api.netbird.io:443
managementURL: "https://my.selfhosted-nb.net:33073"
ingress:
  enabled: true
  router:
    enabled: true  
    replicas: 1   
  kubernetesAPI:
    enabled: false 
operator:
  # This will set the replicaset count more information can be found here: https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/
  replicaCount: 1

cluster:
  # Cluster DNS name (used for webhooks certificates and for network resource DNS names)
  dns: svc.cluster.local
  # Cluster name (used for generating network and network resource names in NetBird)
  name: prod-my-kubernetes

netbirdAPI:
  keyFromSecret: 
       name: "netbird-mgmt-api-key"
       key: "NB_API_KEY"
       
routingClientImage: netbirdio/netbird:rootless-latest

Logs:

2025-07-31T08:20:39Z	INFO	NBGroup	group still linked to resources on netbird	{"namespace": "monitoring", "name": "router", "err": "group has been linked to setup key: kubernetes"}
2025-07-31T08:20:39Z	INFO	Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler	{"controller": "nbgroup", "controllerGroup": "netbird.io", "controllerKind": "NBGroup", "NBGroup": {"name":"router","namespace":"monitoring"}, "namespace": "monitoring", "name": "router", "reconcileID": "b1cab3fe-5ddb-4a43-b1d8-015a3f4ba775"}
2025-07-31T08:20:39Z	ERROR	Reconciler error	{"controller": "nbgroup", "controllerGroup": "netbird.io", "controllerKind": "NBGroup", "NBGroup": {"name":"router","namespace":"monitoring"}, "namespace": "monitoring", "name": "router", "reconcileID": "b1cab3fe-5ddb-4a43-b1d8-015a3f4ba775", "error": "group has been linked to setup key: kubernetes"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.0/pkg/internal/controller/controller.go:332
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.0/pkg/internal/controller/controller.go:279
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.0/pkg/internal/controller/controller.go:240

[mohamed-essam]

Apologies for the late response, I can see you changed cluster.name in your values file, the Group Name for the routing peer should be based on that value (as seen here), was the operator installed using that value initially or did you attempt to change it after installation?

[zeinoux]

Apologies for the late response, I can see you changed cluster.name in your values file, the Group Name for the routing peer should be based on that value (as seen here), was the operator installed using that value initially or did you attempt to change it after installation?

Previously i initiated in my cluster without declaring

cluster:
  name: prod-my-kubernetes

So, after that i tried to uninstall helm release, and declare it into my custom value and then reinstall, and finally it was not affected for peer group name.

Is there a possibility that there are bugs in the helm templating?

EDIT:

It only affects the network resource name and setup key name, the peer group name remains "kubernetes"

[zeinoux]

Hello @mohamed-essam, this case is still happened for my cluster. Any update to fixing this issue?

[zeinoux]

Again is happened when i try installing to another EKS cluster

[mohamed-essam]

Hi @zeinoux , apologies for the late response,

I've just tested on a fresh installation and the Group was named correctly after cluster.name in helm.

Keep in mind that changing the cluster name after installation will not affect the NBGroup name as its immutable, you can reset this behavior by deleting the NBRoutingPeer (which in turn will delete the Network, Network Resources, and the Routing Peer configuration), and then re-applying it, this should re-create the NBGroup correctly and resolve your issue.

For future reference, I will try to see if I can remove the immutability of the Group name and handle this issue more gracefully from the operator side.