Skip document_already_exists_exception logging

[gmoskovicz]

When a document already exists, and we do not want to update the content, we should use the create method in the elasticsearch output. This fils up the disk when a lot of documents already exists. In case someone doesn't care about this being logged, we should check the status, and the type of the exception to avoid logging this type of exceptions:

 {:status=>409, .................................... "error"=>{"type"=>"document_already_exists_exception", "reason"=>"[facebook][321770180859_10153425431690860]: document already exists", "shard"=>"0", "index"=>"social"}}}, :level=>:warn}

I propose a new configuration called skip_error which takes a string that is the error type (in this case document_already_exists_exception) so we don't log it.

[gmoskovicz]

Closed by #424

[SuperFunks]

This feature doesnt seem to work in logstash 5.2.2 - also it is not clear if its the statuscode you "whitelist" to blacklist it from the logoutput or if its the error type string.
Tried both - errors like these are still persisted in the log - filling up the disk.

[2017-03-28T07:45:10,263][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$4@5ccf9d0f on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@4c64817c[Running, pool size = 18, active threads = 18, queued tasks = 562, completed tasks = 47167573]]"})

[gmoskovicz]

@SuperFunks may be broken in 5.2.2 , but you should be able to add the specific error, in this case es_rejected_execution_exception in the whitelist.

The documentation is here: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-failure_type_logging_whitelist . And the test with the example is here: https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/424/files#diff-fcc0c3ab23ddc8b47ed814f1a7b88ea4R45.