feat: HBase Listener integration

[adwk67]

Description

fixes: #618

Implementation notes

• Ephemeral listener volumes are used for non-externally-reachable endpoints by calling add_listener_volume_by_listener_class (unlike HDFS, where the NameNodes have a persistent listener volume)
• Persistent volumes are used for any user-facing (i.e. written to a ConfigMap) endpoints
• unlike Kafka (where the bootstrap listener is constructed by the operator), but like HDFS (where the listener-operator creates the listenters based on the PVCs), the listener references have to be fetched from k8s: this is done using a similiar pattern to HDFS (see mod.rs: functions rolegroup_ref, rolegroup_ref_and_replicas, pod_refs, listener_refs)
• a config map named *-ui-endpoints (e.g. test-hbase-ui-endpoints) is created, containing an endpoint for one replica in each role, as long as the listener-class is externally reachable e.g.

  • apiVersion: v1
    data:
      test-hbase-master-default-0.http: 172.19.0.6:30538
      test-hbase-master-default-1.http: 172.19.0.3:31900                                                                                                                                                          
      test-hbase-regionserver-default-0.http: 172.19.0.6:30482                                                                                                                                                    
      test-hbase-restserver-external-unstable-0.http: 172.19.0.6:32137  
    kind: ConfigMap
    
• these endpoints are checked in the test, which tests different listener classes, resolved across rolegroups/role.

CRD change

Note

There does not need to be a decision on this (discussed briefly in planning 09.04.2024) as the implementation is in line with existing implementations for HDFs and Kafka.

old:

clusterConfig:
   listenerClass: {external-unstable|external-stable|cluster-internal}

new:
(this is inline with the implementations for HDFS and Kafka)

spec:
  masters:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}
  regionServers:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}
  restServers:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}

Tests (OpenShift)

--- PASS: kuttl (1736.72s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/cluster-operation_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (251.16s)
        --- PASS: kuttl/harness/omid_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true_omid-1.1.2 (287.90s)
        --- PASS: kuttl/harness/orphaned_resources_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (186.60s)
        --- PASS: kuttl/harness/smoke_hbase-2.6.1_hdfs-3.4.1_zookeeper-3.9.3_listener-class-external-unstable_openshift-true (320.82s)
        --- PASS: kuttl/harness/overrides_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (182.74s)
        --- PASS: kuttl/harness/kerberos_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_listener-class-external-unstable_kerberos-realm-PROD.MYCORP_kerberos-backend-mit_openshift-true (510.35s)
        --- PASS: kuttl/harness/snapshot-export_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (259.22s)
        --- PASS: kuttl/harness/profiling_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (196.50s)
        --- PASS: kuttl/harness/opa_hbase-opa-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_opa-1.0.1_openshift-true (280.21s)
        --- PASS: kuttl/harness/logging_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (209.32s)
        --- PASS: kuttl/harness/shutdown_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (331.83s)
        --- PASS: kuttl/harness/external-access_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_listener-class-external-unstable_openshift-true (180.55s)
        --- PASS: kuttl/harness/resources_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (213.95s)
PASS

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

Preview Give feedback

1. Changes are OpenShift compatible
2. CRD changes approved
3. CRD documentation for all fields, following the style guide.
4. Helm chart can be installed and deployed operator works
5. Integration tests passed (for non trivial changes)
6. Changes need to be "offline" compatible

Reviewer

Preview Give feedback

1. Code contains useful comments
2. Code contains useful logging statements
3. (Integration-)Test cases added
4. Documentation added or updated. Follows the style guide.
5. Changelog updated
6. Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

Preview Give feedback

1. Feature Tracker has been updated
2. Proper release label has been added
3. Roadmap has been updated

[sbernauer]

Can you please run cargo update -p tokio to fix the cargo deny? :)