each pod has only itselfs as a member

[cmeissner]

Actually all hazelcast pods only have one member, itself. So there is an error in logs:

java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/default/endpoints/hazelcast
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_131]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_131]
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_131]
	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:801) [spring-boot-1.4.6.RELEASE.jar!/:1.4.6.RELEASE]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:785) [spring-boot-1.4.6.RELEASE.jar!/:1.4.6.RELEASE]
	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:772) [spring-boot-1.4.6.RELEASE.jar!/:1.4.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:317) [spring-boot-1.4.6.RELEASE.jar!/:1.4.6.RELEASE]
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.6.RELEASE.jar!/:1.4.6.RELEASE]
	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [bootstrapper.jar:na]

I assume that this happens because we use in our cluster RBAC and so it's not so easy to query the api server. Here we need some configuration. May be we also need to define a RoleBinding.

I already try this:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: hazelcast
  name: hazelcast
--- 
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: hazelcast-minimal
rules:
- apiGroups: [""]
  resources: 
  - endpoints
  verbs: 
  - get
  - watch
  - list
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata: 
  name: hazelcast-minimal
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: hazelcast-minimal
subjects:
- kind: ServiceAccount
  name: hazelcast

But I have no idea to overhand these account to the service.

Cheers
Christian

[pires]

The error means that the app was not able to authenticate against the Kubernetes API server, maybe because there's authorization turned on and this repo doen't support it.

[cmeissner]

Yeah, I think so. But is there a plan to implement support for RBAC secured k8s clusters?

[pires]

It's been years since I last used Hazelcast, so I can't commit to do it. I believe @noctarius (Chris) had an alternative approach that may just work for you. An operator for Hazelcast would be a great project, though.

[noctarius]

Hazelcast has official support for kubernetes as a discovery plugin. See https://github.com/hazelcast/hazelcast-kubernetes/blob/master/README.adoc

I recommend to use the service lookup since DNS discovery is still pretty flanky.