Azure IoT Hub / Azure Event Hub input plugin (with go modules)

[R290]

As discussed in issue #5231 and pull request #5370.
Feedback on code quality or anything else is greatly appreciated.

Required for all PRs:

• Signed CLA.
• Associated README.md updated.
• Has appropriate unit tests.

[sjwang90]

Hi @R290, we will review this PR this week.

[danielnelson]

I haven't ran the plugin yet, planning to try it out soon, but here are my initial comments. In addition to these, I notice there are no tests. This is probably a tricky plugin to unit test, but I think we should get started with at least some tests for Init(), I may have some suggestions later on for more tests too.

[danielnelson]

The metric tracking code is a bit fidgety, sorry about that. I'll put together a change that addresses my comments and open a PR against your fork.

metric tracking comment: #6928 (comment)

[danielnelson]

I opened the PR against your fork, R290#1, which should take care of the metric handling issues. There are a few more things on top of that PR that we need to take care of:

1. I noticed during testing that if you are consuming more than one partition, it isn't safe to use the same offset to restart the consumer. This is because one partition could have a new offset that would be invalid on the other partition. I believe for the starting_offset option we will need to either allow it to be set per offset. Perhaps we should drop this option for now?
2. Still need to add some tests.
3. Rename to eventhub_consumer, we can do this after merge though.

[R290]

Thank you for your pull request!

1. You are right and the from starting offset option is dropped
2. What could we test without actually connecting to an eventhub? In an earlier comment you mentioned testing the Init() function, how would one go about writing tests for this? (I'm still very green when it comes to writing tests)
3. Fine by me

At some point we could consider dropping the Azure/azure-event-hubs-go library altogether and creating a generic AMQP 1.0 consumer based on Azure/go-amqp. Some elements from Azure/azure-amqp-common-go could be needed for the Event Hub and IoT Hub specific stuff. I think this would allow for a plugin closer to the Telegraf design philosophy?

[danielnelson]

I rebased your branch since I couldn't figure out why GitHub still reported a conflict.

At some point we could consider dropping the Azure/azure-event-hubs-go library altogether and creating a generic AMQP 1.0 consumer based on Azure/go-amqp.

I'd definitely like to do have a generic amqp1 input. I'm imagining we would have an eventhub_consumer + amqp1_consumer inputs and an eventhub + amqp1 output. I'm not sure if we could get away with eventually having only the amqp1 plugins though, we might need the eventhub plugin for some of the authentication options?