Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Remove content from telemetry component docs and refer to new module docs #18042

Merged
merged 5 commits into from
Aug 29, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/01-overview/_sidebar.md
Original file line number Diff line number Diff line change
@@ -7,5 +7,4 @@
* [Serverless](/01-overview/serverless/README.md)
* [Service Management](/01-overview/service-management/README.md)
* [Service Mesh](/01-overview/service-mesh/README.md)
* [Telemetry](/01-overview/telemetry/README.md)
* [User Interfaces](/01-overview/ui/README.md)
58 changes: 1 addition & 57 deletions docs/01-overview/telemetry/README.md
Original file line number Diff line number Diff line change
@@ -2,60 +2,4 @@
title: What is Telemetry in Kyma?
---

Fundamentally, ["Observability"](https://opentelemetry.io/docs/concepts/observability-primer/) is a measure of how well the application's external outputs can reflect the internal states of single components. The insights that an application and the surrounding infrastructure expose are displayed in the form of metrics, traces, and logs - collectively, that's called "telemetry" or ["signals"](https://opentelemetry.io/docs/concepts/signals/). These can be exposed by employing modern instrumentation.

![Stages of Observability](./assets/telemetry-stages.drawio.svg)

1. In order to implement Day-2 operations for a distributed application running in a container runtime, the single components of an application must expose these signals by employing modern instrumentation.
2. Furthermore, the signals must be collected and enriched with the infrastructural metadata in order to ship them to a target system.
3. Instead of providing a one-size-for-all backend solution, Kyma supports you with instrumenting and shipping your telemetry data in a vendor-neutral way.
4. This way, you can conveniently enable observability for your application by integrating it into your existing or desired backends. Pick your favorite among many observability backends, available either as a service or as a self-manageable solution, that focus on different aspects and scenarios.

Kyma's Telemetry module focuses exactly on the aspects of instrumentation, collection, and shipment that happen in the runtime and explicitly defocuses on backends.

> **TIP:** An enterprise-grade setup demands a central solution outside the cluster, so we recommend in-cluster solutions only for testing purposes. If you want to install lightweight in-cluster backends for demo or development purposes, check the [Telemetry tutorials](/docs/user/05-tutorials.md).
## Features

To support telemetry for your applications, Kyma's Telemetry module provides the following features:

- Guidance for the instrumentation: Based on [Open Telemetry](https://opentelemetry.io/), you get community samples on how to instrument your code using the [Open Telemetry SDKs](https://opentelemetry.io/docs/instrumentation/) in nearly every programming language.
- Tooling for collection, filtering, and shipment: Based on the [Open Telemetry Collector](https://opentelemetry.io/docs/collector/), you can configure basic pipelines to filter and ship telemetry data.
- Integration in a vendor-neutral way to a vendor-specific observability system: Based on the [OpenTelemetry protocol (OTLP)](https://opentelemetry.io/docs/reference/specification/protocol/), you can integrate backend systems.
- Opt-out from features for advanced scenarios: At any time, you can opt out for each data type, and use custom tooling to collect and ship the telemetry data.
- SAP BTP as first-class integration: Integration into BTP Observability services is prioritized.
- Enterprise-grade qualities - the setup is battle-tested and will satisfy typical development standards.

Initially,
- metrics will not be supported; follow the related [epic](https://github.com/kyma-project/kyma/issues/13079) for tracking the progress of the minimal first version
- logs will not be based on the vendor-neutral OTLP protocol; follow this [epic](https://github.com/kyma-project/kyma/issues/16307) to understand the current progress on that
- the focus on filtering capabilities is to reduce the overall shipment bandwidth to an acceptable minimum, because telemetry data is high-volume data that is always related to a price. So filtering features focus on reducing signal data to, for example, relevant Namespaces or workloads. By default, system-related telemetry data will not be shipped. For custom scenarios, there is always the opt-out possibility.

## Scope

The Telemetry module focuses only on the signals of application logs, distributed traces, and metrics. Other kinds of signals are not considered. Also, logs like audit logs or operational logs are not in scope.

Supported integration scenarios are neutral to the vendor of the target system.

## Components

![Components](./assets/telemetry-components.drawio.svg)

### Telemetry Manager

Kyma's Telemetry module ships Telemetry Manager as its core component. Telemetry Manager is a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) that implements the Kubernetes controller pattern and manages the whole lifecycle of all other components covered in the Telemetry module. Telemetry Manager watches for the user-created Kubernetes resources: LogPipeline, TracePipeline, and, in the future, MetricPipeline. In these resources, you specify what data of a signal type to collect and where to ship it.
If Telemetry Manager detects a configuration, it rolls out the relevant components on demand.

For more information, see [Telemetry Manager](./telemetry-01-operator.md).

### Log Collector

The log agent is based on a [Fluent Bit](https://fluentbit.io/) installation running as a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/). It reads all containers' logs in the runtime and ships them according to a LogPipeline configuration.

For more information, see [Logs](./telemetry-02-logs.md).

### Trace Collector

The Trace Collector is based on an [OTel Collector](https://opentelemetry.io/docs/collector/) [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/). It provides an [OTLP-based](https://opentelemetry.io/docs/reference/specification/protocol/) endpoint to which applications can push the trace signals. According to a TracePipeline configuration, the gateway processes and ships the trace data to a target system.

For more information, see [Traces](./telemetry-03-traces.md).
This content has been moved to: [What is Telemetry in Kyma?](https://kyma-project.io/#/telemetry-manager/user/README).
4 changes: 0 additions & 4 deletions docs/01-overview/telemetry/assets/logging-arch.drawio.svg

This file was deleted.

4 changes: 0 additions & 4 deletions docs/01-overview/telemetry/assets/logging-flow.drawio.svg

This file was deleted.

385 changes: 0 additions & 385 deletions docs/01-overview/telemetry/assets/logging-pipelines.drawio.svg

This file was deleted.

474 changes: 0 additions & 474 deletions docs/01-overview/telemetry/assets/logging-resources.drawio.svg

This file was deleted.

This file was deleted.

328 changes: 0 additions & 328 deletions docs/01-overview/telemetry/assets/operator-lifecycle.drawio.svg

This file was deleted.

This file was deleted.

This file was deleted.

4 changes: 0 additions & 4 deletions docs/01-overview/telemetry/assets/tracing-arch.drawio.svg

This file was deleted.

4 changes: 0 additions & 4 deletions docs/01-overview/telemetry/assets/tracing-intro.drawio.svg

This file was deleted.

448 changes: 0 additions & 448 deletions docs/01-overview/telemetry/assets/tracing-resources.drawio.svg

This file was deleted.

15 changes: 0 additions & 15 deletions docs/01-overview/telemetry/telemetry-01-operator.md

This file was deleted.

462 changes: 0 additions & 462 deletions docs/01-overview/telemetry/telemetry-02-logs.md

This file was deleted.

353 changes: 0 additions & 353 deletions docs/01-overview/telemetry/telemetry-03-traces.md

This file was deleted.

11 changes: 0 additions & 11 deletions docs/01-overview/telemetry/telemetry-04-tutorials.md

This file was deleted.

3 changes: 0 additions & 3 deletions docs/05-technical-reference/00-custom-resources/_sidebar.md
Original file line number Diff line number Diff line change
@@ -7,6 +7,3 @@
* [EventingBackend](evnt-02-eventingbackend.md)
* [Istio](oper-01-istio.md)
* [Function](svls-01-function.md)
* [LogPipeline](telemetry-01-logpipeline.md)
* [LogParser](telemetry-02-logparser.md)
* [TracePipeline](telemetry-03-tracepipeline.md)

This file was deleted.

This file was deleted.

This file was deleted.

17 changes: 1 addition & 16 deletions hack/table-gen/Makefile
Original file line number Diff line number Diff line change
@@ -1,20 +1,5 @@
.PHONY: generate
generate: telemetry-docs cra-docs eventing-docs apix-docs istio-docs

.PHONY: telemetry-tracepipeline
telemetry-tracepipeline:
go run main.go --crd-filename ../../installation/resources/crds/telemetry/tracepipelines.crd.yaml --md-filename ../../docs/05-technical-reference/00-custom-resources/telemetry-03-tracepipeline.md

.PHONY: telemetry-logpipeline
telemetry-logpipeline:
go run main.go --crd-filename ../../installation/resources/crds/telemetry/logpipelines.crd.yaml --md-filename ../../docs/05-technical-reference/00-custom-resources/telemetry-01-logpipeline.md

.PHONY: telemetry-logparser
telemetry-logparser:
go run main.go --crd-filename ../../installation/resources/crds/telemetry/logparsers.crd.yaml --md-filename ../../docs/05-technical-reference/00-custom-resources/telemetry-02-logparser.md

.PHONY: telemetry-docs
telemetry-docs: telemetry-tracepipeline telemetry-logparser telemetry-logpipeline
generate: cra-docs eventing-docs apix-docs istio-docs

.PHONY: cra-docs
cra-docs: