Receiver Node Documentation Request

[darth-drew]

Team,

I am spinning up a receiver node now to test in a dev environment, but I am hoping for some documentation updates on the following:

• Hardware Requirements/Recommendations based on EPS (or some more appropriate metric)
• Architecture diagram depicting receiver node type
• How it fits in when deploying to an existing distributed deployment (is all traffic sent to the receiver after it joins the grid, or only after a manager goes down, etc.)
• Are multiple receiver nodes supported/recommended? If so, are logs load balanced between multiple receiver nodes?
• WRT custom logstash pipelines configured on the manager, I assume these will have to be copied to the receiver nodes sls when installation completes.. assuming of course that logs are going to be sent through the receiver node without other manual steps being taken first

Thank you for adding this node type! It should alleviate a number of concerns for my team.

[TOoSmOotH]

Receiver nodes are running 2 main services which are Logstash and REDIS. They are acting as an event router and don't require a ton of CPU or disk however the more RAM the better for your REDIS queue size similar to the manager. When a receiver node joins the grid, filebeat on all nodes adds this new address as a load balanced logstash output. The search nodes add this new node as another logstash input. Receiver nodes are "active-active" and you can add as many as you want and events will be balanced among them. I am sure there are limitations though with filebeat and logstash as far as number of input/output entries. If you are cloning events on the manager and sending to say an external syslog collector then you would need to replicate that configuration on the receiver nodes as well.

Receiver nodes main goal is to add redundancy to the data pipeline in case the manager goes down. Previously if the manager went down the nodes would queue events locally for a certain period of time and then shed those events eventually since they cannot be sent. Now if the manager goes down all the events keep flowing to the receiver nodes and then get ingested by the search nodes.

I would only recommend receiver nodes if you are in a full distributed deployment and need this type of redundancy. You should be able to add them on the fly to an existing deployment.

[darth-drew]

Thanks Mike, appreciate the info. We are running a full distributed deployment (true clustering) and require this type of redundancy.

The receiver node installed successfully. What have you found is the best way to check how well the events are being balanced? curl -s localhost:9600/_node/stats | jq .pipelines.manager and curl -s localhost:9600/_node/stats | jq .pipelines.receiver on that receiver nodes after restarting logstash?

[dougburks]

Updated the following based on Mike's response:
https://docs.securityonion.net/en/2.3/architecture.html#receiver-node
https://docs.securityonion.net/en/2.3/hardware.html#receiver-node