Log events don't arrive in Elasticsearch with {@Property} in the Messagetemplate

[likemike91]

I use the Elasticsearch sink (DurableElasticsearchSink) in an IIS hosted WCF service.

Question 1: Is this a supported scenario?

Question 2: Some of my log messages aren't sent to Elasticsearch when the Message Template includes an {@MyClassValues} but in the BufferBaseFile-Bookmark file the bookmark is set to the last line. (The shipper "thinks" it has already sent the log event but I don't see it in Elasticsearch/Kibana).
An interesting thing is that some other log messages with @ destructuring operators work as expected.

The object I want to destructure is an instance of the following class:

public class MyClass
{
    public Guid Identifier { get; set; }

    public int MyInt1 { get; set; }

    public int MyInt2 { get; set; }
}

Are there any known issues?

[likemike91]

Now I've an update to that question:
I always receive an error log entry in Elasticsearch with the following message:

failed to execute bulk item (index) index {[myIndex-2016.07.04][logevent][AVW0dKEFD6K2nmshipFf], source[{"@timestamp":"2016-07-04T05:49:24.6961814+00:00","level":"Information","messageTemplate":"MyComplexProperty: {@MyComplexProperty}","fields":{"MyComplexProperty":{"_typeTag":"MyClass","Identifier":"9b4ba4c9-1b95-4693-8d42-af519102db9a","MyInt1":13,"MyInt2":0}}}]}
MapperParsingException[failed to parse [fields.MyComplexProperty]]; nested: IllegalArgumentException[unknown property [_typeTag]];
    at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:329)
    at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:309)
    at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:326)
    at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:252)
    at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:306)
    at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:326)
    at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:252)
    at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:122)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
    at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:529)
    at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.java:506)
    at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:215)
    at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
    at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
    at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
    at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException: unknown property [_typeTag]
    at org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateFieldForString(StringFieldMapper.java:362)
    at org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateField(StringFieldMapper.java:311)
    at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321)
    ... 25 more

Maybe someone can tell me what's the problem here?

[likemike91]

Ok I've found the solution myself.
The problem was that my Index in elasticsearch had a wrong field mapping (string instead of object) and therefore it wasn't working.

[mivano]

Sorry for not responding earlier. Yes, mapping issues are tricky. We need to improve the way we get errors from ES and somehow surface them. The selflog wont show those at the moment, you need the logs from ES itself.

[samchan22]

how will serilog handle if the BULK of elastic returning Error = true ?