Can only have 1 master, which is not a good practice

[xoco70]

This is a great tutorial, thanks!

Now it is limited to have only 1 master isn't it ?

In EK doc, it states:

The minimum_master_nodes setting is extremely important to the stability of your cluster. This setting helps prevent split brains, the existence of two masters in a single cluster.

When you have a split brain, your cluster is at danger of losing data. Because the master is considered the supreme ruler of the cluster, it decides when new indices can be created, how shards are moved, and so forth. If you have two masters, data integrity becomes perilous, since you have two nodes that think they are in charge.

This setting tells Elasticsearch to not elect a master unless there are enough master-eligible nodes available. Only then will an election take place.

This setting should always be configured to a quorum (majority) of your master-eligible nodes. A quorum is (number of master-eligible nodes / 2) + 1. Here are some examples:

How should we create a cluster with multiple masters ?

[yoanisgil]

Yeah, one master is not a good practice. Let me look into it :).

[yoanisgil]

According to this by default all nodes are master-eligible. Should our master go down I think we can point the cluster to another node and we should be able to go (that should give enough time to fix/replace the master and be back in the game).

Now, this is all not too safe so if we're talking about a production scenario then we need to split node roles and have dedicated master and data nodes. That should still be doable with docker-compose but it requires a more complex setup. I'm happy to jump into a more detailed discussion if you're heading that way.

[xoco70]

Yes off course. The difficult thing in your config is:

es-node:
  image: elasticsearch:2
  command: elasticsearch --discovery.zen.ping.unicast.hosts=es_master

In this point, you are able to use unicast.host because you have a single node, but how should you point it when having multiple master nodes.

My guess for the config, was to use the discovery.zen.minimum_master_nodes to define how many master minimum do we need.

But as Swarm IP are random, we cannot give a list of IP in unicast.hosts...

[yoanisgil]

One immediate fix for that is to create 3 master nodes with different names, like es_master1, es_master2 and es_master3. Then it becomes a matter of putting the list of hosts as part of the config. Please note that any node in the ES cluster can act as a master (if it's configuration allows it).

Also take a look at this towards the end of the file you'll see it suggests that you should put a list of master nodes and call it a day :).

[xoco70]

Yep. It could be a static solution. I will look for a dynamic solution and tell you when I get it :)

[yoanisgil]

Multicast is one such but then it's not recommended for production setups. Let me know what you find out.