ECK 2.14.0: adding resources requests will cause ES to ignore the affinity configuration

[hajali-amine]

The resource request block when added, will block the provisioning of the cluster because the affinity configuration is not taken in consideration.

Once we delete the resource block - all is resolved and works fine.

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: es-cluster
spec:
  version: 8.12.2
  secureSettings:
  - secretName: aws-credentials
    entries:
      - key: s3.client.default.access_key
      - key: s3.client.default.secret_key

  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 20Gi
              cpu: "3.5"
            limits:
              memory: 25Gi

        tolerations:
        - key: "es"
          operator: "Equal"
          value: "cluster"
          effect: "NoSchedule"

        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
              - matchExpressions:
                - key: es
                  operator: In
                  values:
                  - cluster
          podAntiAffinity:
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    elasticsearch.k8s.elastic.co/cluster-name: es-cluster
                topologyKey: kubernetes.io/hostname