[ingest] Creates the simulate api route

[BigFunger]

Replaces #6071 (1 of 3)
Creates the server-side functionality for node ingest simulate.

• Adds keysDeep function and tests
  • keysDeep flattens a nested object into a list of properties
  • Uses dot notation for properties
  • Arrays are not exploded
• adds 'set' ingest_processor_type
  • initial processor type to demonstrate the structure of a processor type
• adds /api/kibana/ingest/simulate route
  • POST action endpoint
  • accepts a populated pipeline and returns a set of results for each processor in the submitted pipeline
• adds ingest_simulate module and tests. The module contains two functions.
  • buildRequest packages pipeline into a format that elasticsearch expects
  • processResponse prepares a response for each processor in the submitted pipeline

Pipeline

The simulate route expects a form payload of a pipeline object in JSON.

{
  input: {}, //the sample object that will be the input for the first processor in the pipeline
  processors : [ /* an array of processor objects. There structure is defined below */ ]
}

Processor

The pipeline object contains an array of one or more processor objects.

{
  processorId: 'processor1', //this is a string that uniquely identifies the processor in the pipeline
  typeId: 'set', //this is a string that matches the typeId of it's processor_type object.
  ... //the processor can have any number of other properties that are important for that processor type more information in the processor type section.
}

Processor Type

The processor types are a co-ordinate scope collection. The client will instantiate an instance of a processor_type to create a Processor object. Each processor type will contain the common structure shown below along with default values for any properties that are important for that processor type. For example, a 'set' processor type needs to know the name of the field that is going to be added 'targetField' and the value of that new field 'value'. The 'split' processor on the other hand needs a 'sourceField' property that holds the name of an existing field, and a 'separator' property that will be used to split the value of the sourceField.

{
  typeId: 'set', //this is a unique value that identifies the processor type. This will also be used by the client to determine the directive name to render the corresponding ui.
  title: 'Set', //this is the user-friendly title for the processor type. This is used in any messaging to the user in the ui.
  getDefinition: function(processor) { ... }, //takes a processor that is an instance of that processor type and generating an object representation of that processor in the format that elasticsearch expects. ***
  getDescription: function(processor) { ... } //takes a processor that is an instance of this processor type and generates a string that is a quick summary of the action that processor is taking. This is used in the title-bar of the processor in the ui.
}

*** It is important that each property of a processor type that is used in getDefinition be set to a value in the processor type. Failing to do so will cause unexpected results in the ui.

[Bargs]

API document needs to be snake cased to follow Kibana API conventions as discussed in #5429.

[Bargs]

@BigFunger were you using Postman while developing this, and if so do you happen to have a collection of examples you could share via the share link functionality in Postman?

[Bargs]

Waiting on feedback here: BigFunger#6

[BigFunger]

link to ingest documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html