Multi Node Support for ElasticSearch

[knechtionscoding]

Do we want to support Multi-node installations for Elastic Search? Because I can make the PR, including the updating the FW role that was just added, but only if you want to have that functionality.

it would involve separating out each of the roles on the server side into Kibana, logstash, elasticsearch. It'd also involve separating out the FW, or at least delegating them to the correct place using when conditions.

[sadsfae]

Hey @knechtionscoding I think this would be a great enhancement. I do feel it should be optional and not the default however, configurable via install/group_vars/all.yml. The main reason is that most usage for this playbook is a very quick, easy-to-consume all in one deployment.

I've started thinking about this too and the best way to approach it without adding too much additional complexity on the user end.

Anything you'd care to mock up and have time for would be welcome, we could also consider shipping a multinode branch? Though this functionality should really be a configurable option and done in a way that yaml config options set simply drive it and keep the level of idempotence we have currently.

Thanks for proposing this and all the work you've done so far in improving the stack as a whole.

[knechtionscoding]

Let submit a PR with a quick mockup of what I had before when I did multinode setups and then we can chat about how to achieve it.

I think it can be achieved without a multi node branch. I think it could be configurable easily with the YAML options.

No problem. Glad to be contributing back when I use something in a professional capacity and can help out.

[knechtionscoding]

Submitted #64 so we can continue the conversation of what the logic looks like. Once that is done the config files are easy.

[sadsfae]

It's been a while since there's been an update here. I am still planning on getting this implemented it will just need to come second after #66 (Upgrade to 6.4+).

Some notes on multi-node architecture (borrowed from a friend who works at Elastic):

Ideal design is 3 master-eligible nodes and set minimum_master_nodes to 2
and number of data nodes is arbitrary, especially on larger clusters you would have the 3 master nodes separate but for a smaller clusters it's fine to combine them with data nodes.

Obviously this should be configurable as much as it makes sense but that's a good, solid design default for multi-node. I'd also like to re-use as much of the all-in-one code as possible, this may best be in the form of an all.yml config value set for which other nodes should be added, at this point it will still deploy ELK on one specific node but go back and configure any additional nodes as master/data nodes.

One thing to consider here might be setting your default number of shards too as a new configuration item but schema options should probably be best left to the user so long as the minimum for cluster operation is in place.