[Meta] Move Observability integrations to GA

[sorantis]

The majority of Observability integrations are currently either experimental or beta. The modules they have been migrated from are predominantly GA. We need to move the existing packages to GA before the 8.0. Below is the list of packages that should be migrated. Similar to the security integrations GA effort, the list is prioritized according to beats modules' usage telemetry. Should there be any hinders that prevent a package from moving to GA please highlight them here.

Integrations

• Docker Release Docker as GA #1571
• Nginx Make Nginx GA #1614
• Kubernetes Make Kubernetes GA #1610
• Windows Custom Event Logs GA winlog integration #1716
• Kafka Make Kafka GA #1609
• Zookeeper Make Zookeeper GA #1622
• Apache Make Apache integration GA #1607
• MySQL Make MySQL GA #1612
• Redis Make Redis GA #1618
• MongoDB Make Mongodb GA #1611
• Postgres Make Postresql GA #1616
• AWS Make AWS ga #1623
• Azure
• GCP [Enhancement] Move GCP integration from experimental to GA #1568 GA gcp integrations #1717
• HAProxy Make HAProxy GA #1608
• traefik Make Traefik GA #1621
• Tomcat Make Tomcat GA #1620
• NATS Make NATS GA #1613
• STAN Make Stan GA #1619
• Nginx Ingress Controller Make Nginx Ingress Controller GA #1615
• RabbitMQ Make RabbitMQ GA #1617
• CEF GA CEF integration #1604
• Cassandra (BETA) Make Cassandra GA #2129

Requirements for moving a package to GA:

• Any package with httpjson input requires:
  • Closed beats/integrations gap on httpjson system tests
• All packages require:
  • Edge processing moved to ingest node pipeline https://github.com/elastic/security-team/issues/791 we decided to remove it in integrations team meeting on July 16th
  • event.original is optional event.original optionality across all packages #777
  • Bump version to >=1.x.x

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[kaiyan-sheng]

@sorantis For GCP integration, it still only includes the log part from Filebeat. We should finish migrating the metric parts first. So maybe the requirements for moving a package to GA should also include: Make sure all data streams are included?

[sorantis]

let's log this as a prerequisite under the GCP bullet above. This applies to all above integrations. Let's track them under each integration's entry in this meta issue.

[sorantis]

@kaiyan-sheng I changed your edit to a sub-bullet. Do we have a GH issue we could link to?

[mukeshelastic]

Hi @sorantis is the list in priority order? If not, can we make it in the order of priority we want to see it completed?

@masci are you and folks on integrations team able to evaluate this list and provide a comments on what work is needed to make these integrations GA and then the feasibility of the number of integrations we are able to make GA in 7.16? I will also send an email to the integrations team about this so that everyone on the team can share their perspective.

[kaiyan-sheng]

@kaiyan-sheng I changed your edit to a sub-bullet. Do we have a GH issue we could link to?

@sorantis We don't have an issue for only migrating the metrics part but here is the main issue for migrating GCP overall.

[endorama]

GCP integration went GA with only logs related data streams. Migration from beats is still in progress

[mukeshelastic]

What is the remaining on the Azure integration to make it GA? For all integrations, we had a meta issue to describe what was needed but not for azure so can't tell what is to be done for azure to make it GA?

cc @ravikesarwani

[masci]

Let's discuss what we're missing for Azure specifically here #2197

Closing this one as done.