BREAKING CHANGE! New default template for ES 5.0

[untergeek]

Do not use this with older versions of Elasticsearch. It will not work.

This template has been tested with the alpha1 release of Elasticsearch 5.0.0

fixes #386

[suyograo]

Lets actually make it 5.0.0 so we immediately know what we're dealing with when a user reports an issue. normally, we want plugins to have their own version life cycle and follow semver, but this may be a special case where it warrants to track ES 5.0. Otherwise it may lead to a lot of confusing users and us

[untergeek]

Right. I set it to 3.0.0 before we discussed it, then didn't change it after.

[talevy]

is this final 5.0.0 version? should we put an alpha or dev here for now?

[talevy]

or 5.0.0-SNAPSHOT

[untergeek]

set it to alpha1 to match the ES release level.

[untergeek]

I set it to alpha1 to match the ES version.

[jpountz]

Should we keep disabling norms? With the new syntax, this would be "_all" : {"norms" : false}

[untergeek]

If I recall correctly, when I tested a new 5.0 compatible template (rather than just hoping the existing template would be adapted on the fly) it didn't work with "norms" : false. I got some error or other.

[jpountz]

Hmm this must be a bug. :( I just tried locally and it worked for me, do you have steps that I can follow to reproduce?

[jpountz]

This looks great.

[pickypg]

I like that it no longer has the boilerplate dynamic templates for the base types, as it's effectively implied (including with doc values) starting in 2.x. Now that this requires 5.0, it makes total sense!

[untergeek]

We are going to re-do this with an improved version of the current template that will be backward compatible. @jpountz has done some awesome things that will automagically translate old templates into new mappings. We will still prune the old "helper" mappings for building your own numeric types, and since doc_values is default, we'll remove the hard-mappings there as well.

[untergeek]

I PUT the template to the 5.0.0 alpha RC1 release and inserted a single document:

curl -XPUT localhost:9200/logstash-2016.03.30/logs/1 -d '
> { "@timestamp" : "2016-03-30T17:45:00.000Z", "message" : "This is a message field" }
> '
{"_index":"logstash-2016.03.30","_type":"logs","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}

When I pulled the mapping, this is what I see:

{
  "logstash-2016.03.30" : {
    "mappings" : {
      "logs" : {
        "_all" : {
          "enabled" : true,
          "norms" : false
        },
        "dynamic_templates" : [ {
          "message_field" : {
            "match" : "message",
            "match_mapping_type" : "string",
            "mapping" : {
              "fielddata" : {
                "format" : "disabled"
              },
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string"
            }
          }
        }, {
          "string_fields" : {
            "match" : "*",
            "match_mapping_type" : "string",
            "mapping" : {
              "fielddata" : {
                "format" : "disabled"
              },
              "fields" : {
                "raw" : {
                  "ignore_above" : 256,
                  "index" : "not_analyzed",
                  "type" : "string"
                }
              },
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string"
            }
          }
        } ],
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "@version" : {
            "type" : "keyword"
          },
          "geoip" : {
            "dynamic" : "true",
            "properties" : {
              "ip" : {
                "type" : "ip"
              },
              "latitude" : {
                "type" : "float"
              },
              "location" : {
                "type" : "geo_point"
              },
              "longitude" : {
                "type" : "float"
              }
            }
          },
          "message" : {
            "type" : "text",
            "norms" : false,
            "fielddata" : false
          }
        }
      },
      "_default_" : {
        "_all" : {
          "enabled" : true,
          "norms" : false
        },
        "dynamic_templates" : [ {
          "message_field" : {
            "match" : "message",
            "match_mapping_type" : "string",
            "mapping" : {
              "fielddata" : {
                "format" : "disabled"
              },
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string"
            }
          }
        }, {
          "string_fields" : {
            "match" : "*",
            "match_mapping_type" : "string",
            "mapping" : {
              "fielddata" : {
                "format" : "disabled"
              },
              "fields" : {
                "raw" : {
                  "ignore_above" : 256,
                  "index" : "not_analyzed",
                  "type" : "string"
                }
              },
              "index" : "analyzed",
              "omit_norms" : true,
              "type" : "string"
            }
          }
        } ],
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "@version" : {
            "type" : "keyword"
          },
          "geoip" : {
            "dynamic" : "true",
            "properties" : {
              "ip" : {
                "type" : "ip"
              },
              "latitude" : {
                "type" : "float"
              },
              "location" : {
                "type" : "geo_point"
              },
              "longitude" : {
                "type" : "float"
              }
            }
          }
        }
      }
    }
  }
}

[suyograo]

LGTM

[suyograo]

@untergeek copied this to #402 caused i had a lot of merge conflicts.

[untergeek]

Closing this in favor of #402 then.