Reduce default pipelining to Logstash to a more tame '2'. This reduce…

…s memory pressure on LS. (#6250)

We do have improvements to reduce memory usage in LS here logstash-plugins/logstash-input-beats#286

However, I think it makes sense to start with this fix here, and ramp up this number on the next major when we have more confidence that 5 is the magic number.

At the end of the day, magic numbers are always problematic, but 5 can cause quite a bit of pressure when there's a high ratio of Beats:Logstash instances

auditbeat/auditbeat.reference.yml

@@ -356,7 +356,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

filebeat/filebeat.reference.yml

@@ -804,7 +804,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

heartbeat/heartbeat.reference.yml

@@ -465,7 +465,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

libbeat/_meta/config.reference.yml

@@ -251,7 +251,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

libbeat/outputs/logstash/config.go

@@ -31,7 +31,7 @@ type Backoff struct {
 var defaultConfig = Config{
 	Port:             5044,
 	LoadBalance:      false,
-	Pipelining:       5,
+	Pipelining:       2,
 	BulkMaxSize:      2048,
 	SlowStart:        false,
 	CompressionLevel: 3,

metricbeat/metricbeat.reference.yml

@@ -740,7 +740,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

packetbeat/packetbeat.reference.yml

@@ -719,7 +719,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`

winlogbeat/winlogbeat.reference.yml

@@ -280,7 +280,7 @@ output.elasticsearch:
 
   # Number of batches to be sent asynchronously to logstash while processing
   # new batches.
-  #pipelining: 5
+  #pipelining: 2
 
   # If enabled only a subset of events in a batch of events is transferred per
   # transaction.  The number of events to be sent increases up to `bulk_max_size`