Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Google Pub/Sub as telemetry data message infrastructure #3442

Closed
julian-sotec opened this issue Nov 15, 2022 · 4 comments
Closed

Support Google Pub/Sub as telemetry data message infrastructure #3442

julian-sotec opened this issue Nov 15, 2022 · 4 comments
Assignees
@matthiasfeurer @julian-sotec
Labels
C&C Command and Control Feature Request A request for adding new functionality to Hono MQTT Adapter
Milestone
2.4.0

Comments

@julian-sotec
Copy link

This is the second issue to qualify Hono as a replacement for Google IoT Core customers. (see #3441)
As part of this architecture we would like to add native Pub/Sub messaging infrastructure support to Hono. We want to adapt the same topic structure that Hono uses for Kafka messaging. The PubSub message should contain the same header information as as on Google IOT Core.
Internal communication between services will continue to use AMQP. Hono does support sending commands to the devices via Kafka messages. This will not be supported in Pub/Sub for now, instead we intend to implement this via REST API.
The Pub/Sub bridge should work foremost with MQTT, maybe later it could be extended with another protocol adapter.

Pub/Sub Messaging:
@sophokles73
Copy link
Contributor

Internal communication between services will continue to use AMQP. Hono does support sending commands to the devices via Kafka messages. This will not be supported in Pub/Sub for now, instead we intend to implement this via REST API.

Would you mind providing a little more detail regarding the REST API that you envision? Would that become a standard Hono API? If so, where do you see this API to be added?

@sophokles73 sophokles73 added MQTT Adapter C&C Command and Control Feature Request A request for adding new functionality to Hono labels Nov 16, 2022
@matthiasfeurer
Copy link
Contributor

Internal communication between services will continue to use AMQP. Hono does support sending commands to the devices via Kafka messages. This will not be supported in Pub/Sub for now, instead we intend to implement this via REST API.

Would you mind providing a little more detail regarding the REST API that you envision? Would that become a standard Hono API? If so, where do you see this API to be added?

We would submit a separate issue for the REST API, but to give a short answer here: this would most probably be a new API talking to the Command Router so that all upstream logic via MQTT Adapter can stay the same as is. Terminology wise we would try to comply with Hono standards, so that it can be included into the core project. We will provide more details in the separate issue.

michelleFranke added a commit to sotec-iot/hono that referenced this issue Dec 9, 2022
@michelleFranke
[eclipse-hono#3442] This is the second issue to qualify Hono as a rep…
5da1446 
…lacement for Google IoT Core customers.

As part of this architecture we added a Pub/Sub based client for downstream messages.

Added new modules to support Google Pub/Sub as messaging infrastructure.

Used Google's TopicAdminClient to create topics.

Used Google's Publisher to publish messages to Google Pub/Sub.

Added unittests.

Signed-off-by: michelle <[email protected]>
michelleFranke added a commit to sotec-iot/hono that referenced this issue Dec 9, 2022
@michelleFranke
[eclipse-hono#3442] Support Google Pub/Sub as messaging infrastructure.
34d8d2b 
This is the second issue to qualify Hono as a replacement for Google IoT Core customers.
As part of this architecture we added a Pub/Sub based client for downstream messages.

Added new modules to support Google Pub/Sub as messaging infrastructure.

Used Google's TopicAdminClient to create topics.

Used Google's Publisher to publish messages to Google Pub/Sub.

Added unittests.

Signed-off-by: michelle <[email protected]>
michelleFranke added a commit to sotec-iot/hono that referenced this issue Jan 25, 2023
@michelleFranke
[eclipse-hono#3442] Support Google Pub/Sub as messaging infrastructure.
21dc422 
This is the second issue to qualify Hono as a replacement for Google IoT Core customers.
As part of this architecture we added a Pub/Sub based client for downstream messages.

Added new modules to support Google Pub/Sub as messaging infrastructure.

Used Google's TopicAdminClient to create topics.

Used Google's Publisher to publish messages to Google Pub/Sub.

Added unittests.

Signed-off-by: michelle <[email protected]>
sophokles73 pushed a commit that referenced this issue Jan 26, 2023
@michelleFranke @sophokles73
[#3442] Support Google Pub/Sub as messaging infrastructure.
593cd58 
This is the second issue to qualify Hono as a replacement for Google IoT Core customers.
As part of this architecture we added a Pub/Sub based client for downstream messages.

Added new modules to support Google Pub/Sub as messaging infrastructure.

Used Google's TopicAdminClient to create topics.

Used Google's Publisher to publish messages to Google Pub/Sub.

Added unittests.

Signed-off-by: michelle <[email protected]>
@sophokles73 sophokles73 added this to the 2.3.0 milestone Jan 26, 2023
sophokles73 added a commit that referenced this issue Jan 28, 2023
@sophokles73
[#3442] Document Google PubSub Client dependencies
16117ca 
Signed-off-by: Kai Hudalla <[email protected]>
sophokles73 added a commit that referenced this issue Jan 30, 2023
@sophokles73
[#3442] Use Quarkiverse Google Cloud Services extension
5ba30bd 
The native image build has been failing ever since the standard Google
Pub/Sub client libraries had been introduced.

The Quarkus extension fixes this problem.

Signed-off-by: Kai Hudalla <[email protected]>
sophokles73 added a commit that referenced this issue Jan 30, 2023
@sophokles73
[#3442] Improve unit tests for Pub/Sub client
339106d 
Signed-off-by: Kai Hudalla <[email protected]>
sophokles73 added a commit that referenced this issue Jan 30, 2023
@sophokles73
[#3442] Do not depend on CredentialsProvider
faa453c 
The protocol adapters failed to start up because the Google Cloud
Services CredentialsProvider could not be created using the default
configuration.

The adapters now depend on the GOOGLE_APPLICATION_CREDENTIALS
environment variable to be set instead.

Signed-off-by: Kai Hudalla <[email protected]>
sophokles73 added a commit that referenced this issue Jan 30, 2023
@sophokles73
[#3442] Update legal docs
0de9e41 
Added missing dependencies.

Signed-off-by: Kai Hudalla <[email protected]>
Merged
@sophokles73 sophokles73 modified the milestones: 2.3.0, 2.4.0 Feb 22, 2023
@michelleFranke michelleFranke mentioned this issue Mar 7, 2023
Closed
sophokles73 added a commit that referenced this issue Mar 14, 2023
@sophokles73
[#3442] Complete message sending future on vert.x event loop
81ae2be 
Signed-off-by: Kai Hudalla <[email protected]>
@sophokles73
Copy link
Contributor

@julian-sotec can this be closed? It looks like the corresponding implementation has been contributed already.

@julian-sotec
Copy link
Author

@julian-sotec can this be closed? It looks like the corresponding implementation has been contributed already.

Done, Thanks for the hint!

@mattkaem mattkaem mentioned this issue May 8, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matthiasfeurer matthiasfeurer

@julian-sotec julian-sotec

Labels
C&C Command and Control Feature Request A request for adding new functionality to Hono MQTT Adapter
Projects
None yet
Milestone
2.4.0
Development

No branches or pull requests
3 participants
@sophokles73 @matthiasfeurer @julian-sotec