Change Default Memory Settings for Elasticsearch and Kibana

[ossie-git]

Hi,

I was kind of surprised that the defaults for both Elasticsearch and Kibana make little sense, even for a test environment. For example, the quickstart example deploys:

• Elasticsearch with a 2GB RAM limit
• Kibana with a 1GB RAM limit

You quickly end up with issues even for testing. For example, while ingesting 100K documents, Logstash would start throwing errors and the port-forward would go down. It turned out that this was due to Elasticsearch running into OOM and restarting. I also ran into issues with Kibana.

While these defaults are mentioned in the documentation, I think that at the very least embedding them into the sample manifests (at the very least) and setting them to something more realistic would be useful. For example:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 8.18.1
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms2g -Xmx2g
          resources:
            requests:
              memory: 4Gi
              cpu: 4
            limits:
              memory: 4Gi

Thanks

[barkbay]

You quickly end up with issues even for testing. For example, while ingesting 100K documents [...]

My feeling is that for a "quick start", we want to keep the resource requests fairly low. I think that by default we want to allow our users to deploy Elasticsearch on local environments like k3d or Kind where we usually have a limited amount of resources. Hundreds of thousands of documents seems a bit beyond the scope of a quick start, at least that's my feeling.

The example here is already setting 4Gi of memory:

        containers:
        - name: elasticsearch
          # specify resource limits and requests
          resources:
            limits:
              memory: 4Gi
              cpu: 1

You don't need to set -Xms2g -Xmx2g.

[ossie-git]

I don't know how 100K documents isn't considered "quick start". It only takes 50MB of disk space or so and I would think that a quick start should be able to store millions of documents. From my understanding, "quick start" = not ready for production because you only have a single node. At the very least, if the team plans on keeping it as is, I would suggest:

• adding the default settings explicitly in the sample YAML so readers can explicitly see the resource limits without having to read additional documentation, especially as they are just testing things out at this stage
• warning users of the limitations of these "quick start" settings

Just my $0.02