No single source of truth for Labels and Taints

[surajssd]

Right now we ask user to specify the labels and taints in the controller/worker pools in a *.lokocfg file. Like following:

  worker_pool "foobar" {
    count = 3
    node_type = "c1.small.x86"

    labels = "foobar.lokomotive.io=contour"
    taints = "foobar.lokomotive.io=contour:NoSchedule"
  }

Now if user wants to change the labels and taints later if they make changes here and run lokoctl cluster deploy it will trigger machine recreation. And since this machine recreation can incur downtime or loss of data.

If the apps are stateless then the node removal should drain the old node, add new one to the cluster and then delete old one.

If the node is running stateful apps then there is no co-ordination with such apps(etcd, ceph, etc) either. Like there also similar process has to be followed add new node, make sure that the app has acknowledged new node & synced all data and then delete old one.

---

So the bottom line is that lokocfg file is not a single source of truth for that information. If user makes changes manually using kubectl then we have a different version in lokocfg state and something else running on the cluster.