Values of fields with names containing dots shown as null

[jimmyjones2]

spark-1.6.2-bin-hadoop2.6, elasticsearch-5.0.0-beta1, elasticsearch-hadoop-5.0.0-beta1

curl -XPOST localhost:9200/test7/test -d '{"a.b":0}'
./bin/pyspark --driver-class-path=../elasticsearch-hadoop-5.0.0-beta1/dist/elasticsearch-hadoop-5.0.0-beta1.jar
>>> df7 = sqlContext.read.format("org.elasticsearch.spark.sql").load("test7/test")
>>> df7.show()
+----+
|   a|
+----+
|null|
+----+

Possibly related to #853

[jbaiera]

Another good find. When we create the scroll query to read data in, we specify each of the fields that we want to retrieve to cut down on transfer costs. The scroll request that ends up getting created here is:

localhost:9200/test4/test/_search?sort=_doc&scroll=5m&size=50&_source=a&preference=_shards:3;_local

The key here is that _source=a parameter. Elasticsearch looks for a field named a on the source values to return, and finds no such field, since the field's name is technically a.b. Thus, the a.b field is filtered out and an empty document is returned.

[jbaiera]

This should be addressed with elastic/elasticsearch#20736, but will still probably have issues with #853