Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(outputs.quix): Add plugin #16144

Merged
merged 23 commits into from
Dec 6, 2024
Merged

Conversation

tomas-quix
Copy link
Contributor

@tomas-quix tomas-quix commented Nov 5, 2024

Summary

Checklist

  • No AI generated code was used in this PR

Related issues

@telegraf-tiger
Copy link
Contributor

telegraf-tiger bot commented Nov 5, 2024

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

@srebhan srebhan changed the title Ouix output plugin feat(outputs.quix): Add plugin Nov 12, 2024
@telegraf-tiger telegraf-tiger bot added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins labels Nov 12, 2024
@srebhan srebhan self-assigned this Nov 12, 2024
@tomas-quix
Copy link
Contributor Author

tomas-quix commented Nov 19, 2024

!signed-cla

1 similar comment
@stereosky
Copy link
Contributor

!signed-cla

@tomas-quix tomas-quix marked this pull request as ready for review November 19, 2024 10:30
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution @tomas-quix! I do have some comments in the code. Furthermore, I want to encourage you to implement some unit-tests for this plugin, e.g. by mocking the config server and using a quix docker container or at least using a kafka instance mocking the replies...

plugins/outputs/quix/README.md Outdated Show resolved Hide resolved
plugins/outputs/quix/README.md Outdated Show resolved Hide resolved
plugins/outputs/quix/README.md Outdated Show resolved Hide resolved
plugins/outputs/quix/README.md Outdated Show resolved Hide resolved
plugins/outputs/quix/sample.conf Outdated Show resolved Hide resolved
plugins/outputs/quix/quix.go Outdated Show resolved Hide resolved
plugins/outputs/quix/quix.go Outdated Show resolved Hide resolved
plugins/outputs/quix/scram_client.go Outdated Show resolved Hide resolved
plugins/outputs/quix/scram_client.go Outdated Show resolved Hide resolved
plugins/outputs/quix/scram_client.go Outdated Show resolved Hide resolved
@srebhan
Copy link
Member

srebhan commented Dec 4, 2024

@tomas-quix I've pushed some cleanups and also added some unit-tests. What is currently missing is an integration test (see commented code in the test file) with a Kafka container configured to use SASL-SCRAM256 which you enforce. Could you take a look please!?!?

Furthermore, I do have the following questions:

  1. The sasl.mechanism parameter of the fetched config is not used at all but the code enforces SASL-SCRAM256. Is this intentional? What should the plugin do if the mechanism says something else?
  2. The security.protocol parameter of the fetched config is not used at all. Is this intentional? If so, why do we pass that around?
  3. Wouldn't it be better to simply define the timestamp_units of the serialized metrics to nanoseconds? This would avoid a potential misconfiguration between the pipeline and Telegraf and the value can be converted to seconds easily on the Quix side, can't it?

@tomas-quix
Copy link
Contributor Author

I have fixed points 1 and 2. I do not understand point 3 however, what do you mean by that? Do you mean to just hardcode nanoseconds as a unit?

@srebhan
Copy link
Member

srebhan commented Dec 5, 2024

I do not understand point 3 however, what do you mean by that? Do you mean to just hardcode nanoseconds as a unit?

@tomas-quix exactly. Why not send the data with the available precision? My problem is that with an option your pipeline needs to know what was configured in Telegraf and vice versa which usually is a source of trouble if both are not maintained by the same person (which is likely)... If you hard-code it, the Quix side can expect the precision/format...

@srebhan srebhan added ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. new plugin labels Dec 5, 2024
@srebhan srebhan assigned DStrand1 and unassigned srebhan Dec 5, 2024
@telegraf-tiger
Copy link
Contributor

telegraf-tiger bot commented Dec 5, 2024

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

⚠️ This pull request increases the Telegraf binary size by 2.23 % for linux amd64 (new size: 262.4 MB, nightly size 256.7 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip
armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip
armhf.deb i386.rpm freebsd_armv7.tar.gz
i386.deb ppc64le.rpm freebsd_i386.tar.gz
mips.deb riscv64.rpm linux_amd64.tar.gz
mipsel.deb s390x.rpm linux_arm64.tar.gz
ppc64el.deb x86_64.rpm linux_armel.tar.gz
riscv64.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_riscv64.tar.gz
linux_s390x.tar.gz

@DStrand1 DStrand1 merged commit a9c91f1 into influxdata:master Dec 6, 2024
29 checks passed
@github-actions github-actions bot added this to the v1.33.0 milestone Dec 6, 2024
justinwwhuang pushed a commit to justinwwhuang/telegraf_fork that referenced this pull request Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin new plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants