Added kafka partition number to metadata.

[davizucon]

Added Kafka partition number to be available to schema metadata __metadata$partition when "stream.kafka.metadata.populate": "true" in table config

Schema snippet:

{
"name": "__metadata$offset",
    ...
},
{
"name": "__metadata$partition",
    ...
},

Related documentation: PR 353 in pinot-docs repo.

[Jackie-Jiang]

The change looks good! Can you elaborate a little bit more on how do you plan to use the partition info? It should be available from the consumer already

[davizucon]

Thanks for the review! If there is any way to get partition info instead this PR, pls point me out.

There is a system where the distribution of messages in the partitions is based on a particular key and algorithm. With partition info we can see how producers are performing. Among other information. Thus we can see if there is any queueing in partitions, which may eventually cause some contentention for consumers.

[Jackie-Jiang]

The consumer is created based on the stream partition, but I guess exposing this info through metadata from the upstream is anyway useful for debugging purpose