From ae85b9e19c5331e33fe44d6081b70b27b92955fd Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Tue, 30 Aug 2022 14:08:14 -0700 Subject: [PATCH 01/20] Add batch.size and separate messaging attributes Add destination name - it could be per-message --- CHANGELOG.md | 20 ++ semantic_conventions/trace/messaging.yaml | 236 +++++++++++------ .../instrumentation/aws-lambda.md | 14 +- .../trace/semantic_conventions/messaging.md | 239 +++++++++++++----- 4 files changed, 362 insertions(+), 147 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4f9c5425ed..5539805ee82 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -133,6 +133,26 @@ release. - Rename `http.retry_count` to `http.resend_count` and clarify its meaning. ([#2743](https://github.com/open-telemetry/opentelemetry-specification/pull/2743)) +- BREAKING: rename `messaging.destination` to `messaging.destination.name`, + `messaging.temp_destination` to `messaging.destination.temporary`, + `messaging.destination_kind` to `messaging.destination.kind`, + `messaging.message_id` to `messaging.message.id`, + `messaging.protocol` to `net.app.protocol.name`, + `messaging.protocol_version`, `net.app.protocol.version`, + `messaging.conversation_id` to `messaging.message.conversation_id`, + `messaging.message_payload_size_bytes` to `messaging.message.payload_size_bytes`, + `messaging.message_payload_compressed_size_bytes` to `messaging.message.payload_compressed_size_bytes`, + `messaging.rabbitmq.routing_key`: `messaging.rabbitmq.message.routing_key`, + `messaging.kafka.message_key` to `messaging.kafka.message.key`, + `messaging.kafka.partition` to `messaging.kafka.message.partition`, + `messaging.kafka.tombstone` to `messaging.kafka.message.tombstone`, + `messaging.rocketmq.message_type` to `messaging.rocketmq.message.type`, + `messaging.rocketmq.message_tag` to `messaging.rocketmq.message.tag`, + `messaging.rocketmq.message_keys` to `messaging.rocketmq.message.keys`; + Removed `messaging.url`; + Renamed `send` operation to `publish`; + Split `destination` and `source` namespaces and clarify per-message attributes in batching scenarios. + ([#2763](https://github.com/open-telemetry/opentelemetry-specification/pull/2763)). ### Metrics diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index ff917a05091..e7273549544 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -1,73 +1,94 @@ groups: - - id: messaging + - id: messaging.message prefix: messaging + # todo (https://github.com/open-telemetry/build-tools/issues/123) type: span - brief: > - This document defines the attributes used in - messaging systems. + brief: 'Semantic convention describing per-message attributes populated on messaging spans or links.' attributes: - - id: system - type: string - requirement_level: required - brief: 'A string identifying the messaging system.' - examples: ['kafka', 'rabbitmq', 'rocketmq', 'activemq', 'AmazonSQS'] - - id: destination + - id: destination.name type: string - requirement_level: required - brief: > - The message destination name. This might be equal to the span name - but is required nevertheless. + brief: 'The message destination name' + note: | + Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If + broker does not have such notion, destination name SHOULD uniquely identify broker. examples: ['MyQueue', 'MyTopic'] - - id: destination_kind - type: - allow_custom_values: false - members: - - id: queue - value: "queue" - brief: "A message sent to a queue" - - id: topic - value: "topic" - brief: "A message sent to a topic" - requirement_level: - conditionally_required: If the message destination is either a `queue` or `topic`. - brief: 'The kind of message destination' - - id: temp_destination - type: boolean - requirement_level: - conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. - brief: 'A boolean that is true if the message destination is temporary.' - - id: protocol - type: string - brief: 'The name of the transport protocol.' - examples: ['AMQP', 'MQTT'] - - id: protocol_version - type: string - brief: 'The version of the transport protocol.' - examples: '0.9.1' - - id: url + - id: source.name type: string - brief: 'Connection string.' - examples: ['tibjmsnaming://localhost:7222', 'https://queue.amazonaws.com/80398EXAMPLE/MyQueue'] - - id: message_id + brief: 'The message source name' + note: | + Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If + broker does not have such notion, source name SHOULD uniquely identify broker. + examples: ['MyQueue', 'MyTopic'] + - id: message.id type: string brief: 'A value used by the messaging system as an identifier for the message, represented as a string.' examples: '452a7c7c7c7048c2f887f61572b18fc2' - - id: conversation_id + - id: message.conversation_id type: string brief: > The [conversation ID](#conversations) identifying the conversation to which the message belongs, represented as a string. Sometimes called "Correlation ID". examples: 'MyConversationId' - - id: message_payload_size_bytes + - id: message.payload_size_bytes type: int brief: > The (uncompressed) size of the message payload in bytes. Also use this attribute if it is unknown whether the compressed or uncompressed payload size is reported. examples: 2738 - - id: message_payload_compressed_size_bytes + - id: message.payload_compressed_size_bytes type: int brief: 'The compressed size of the message payload in bytes.' examples: 2048 + + - id: messaging + prefix: messaging + type: span + brief: > + This document defines the attributes used in + messaging systems. + attributes: + - id: system + type: string + requirement_level: required + brief: 'A string identifying the messaging system.' + examples: ['kafka', 'rabbitmq', 'rocketmq', 'activemq', 'AmazonSQS'] + - id: operation + type: + allow_custom_values: true + members: + - id: publish + value: "publish" + - id: receive + value: "receive" + - id: process + value: "process" + requirement_level: required + brief: > + A string identifying the kind of messaging operation as defined in the + [Operation names](#operation-names) section above. + note: If custom value is used, it MUST known to be of low cardinality. + - id: batch.size + type: int + brief: The number of messages sent, received, or processed in the scope of the batching operation. + requirement_level: + conditionally_required: If the span describes operation on a batch of messages. + note: > + Instrumentations SHOULD NOT set `messaging.batch.size` on spans that operate with a single message. + When client library supports batch and single-message API for the same operation, instrumentations SHOULD + use `messaging.batch.size` for batching APIs and SHOULD NOT use it for single-message APIs. + examples: [0, 1, 2] + - ref: messaging.message.id + requirement_level: + recommended: Only if span represents operation on a single message. + - ref: messaging.message.conversation_id + requirement_level: + recommended: Only if span represents operation on a single message. + - ref: messaging.message.payload_size_bytes + requirement_level: + recommended: Only if span represents operation on a single message. + - ref: messaging.message.payload_compressed_size_bytes + requirement_level: + recommended: Only if span represents operation on a single message. - ref: net.peer.name note: > This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from. @@ -83,7 +104,9 @@ groups: tag: connection-level requirement_level: recommended: If different than `net.peer.name` and if `net.sock.peer.addr` is set. - + - ref: net.app.protocol.name + examples: ['amqp', 'mqtt'] + - ref: net.app.protocol.version constraints: - include: network @@ -93,7 +116,47 @@ groups: extends: messaging span_kind: producer brief: 'Semantic convention for producers of messages sent to a messaging systems.' - + attributes: + - ref: messaging.destination.name + requirement_level: + conditionally_required: If one message is being published or if the value applies to all messages in the batch. + - id: destination.kind + type: + allow_custom_values: false + members: + - id: queue + value: "queue" + brief: "A message sent to a queue" + - id: topic + value: "topic" + brief: "A message sent to a topic" + requirement_level: + conditionally_required: If the message destination is either a `queue` or `topic`. + brief: 'The kind of message destination' + - id: destination.template + type: string + requirement_level: + conditionally_required: > + If available. Instrumentations MUST NOT use `messaging.destination.name` as template + unless low-cardinality of destination name is guaranteed. + brief: Low cardinality field representing messaging destination + note: > + Destination names could be constructed from templates. + An example would be a destination name involving a user name or product id. + Although the destination name in this case is of high cardinality, + the underlying template is of low cardinality and can be effectively + used for grouping and aggregation. + examples: ['/customers/{customerId}'] + - id: destination.temporary + type: boolean + requirement_level: + conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. + brief: 'A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed.' + - id: destination.anonymous + type: boolean + requirement_level: + conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. + brief: 'A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name).' - id: messaging.producer.synchronous prefix: messaging type: span @@ -110,19 +173,6 @@ groups: span_kind: consumer brief: 'Semantic convention for a consumer of messages received from a messaging system' attributes: - - id: operation - type: - allow_custom_values: false - members: - - id: receive - value: "receive" - - id: process - value: "process" - brief: > - A string identifying the kind of message consumption as defined in the - [Operation names](#operation-names) section above. - If the operation is "send", this attribute MUST NOT be set, since the - operation can be inferred from the span kind in that case. - id: consumer_id type: string brief: > @@ -131,6 +181,46 @@ groups: `messaging.kafka.consumer_group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. examples: 'mygroup - client-6' + - ref: messaging.source.name + requirement_level: + conditionally_required: If the value applies to all messages in the batch. + - id: source.kind + type: + allow_custom_values: true + members: + - id: queue + value: "queue" + brief: "A message received from a queue" + - id: topic + value: "topic" + brief: "A message received from a topic" + requirement_level: + conditionally_required: If the message source is either a `queue` or `topic`. + brief: 'The kind of message source' + - id: source.template + type: string + brief: 'Low cardinality field representing messaging source' + requirement_level: + conditionally_required: > + If available. Instrumentations MUST NOT use `messaging.source.name` as template + unless low-cardinality of source name is guaranteed. + examples: ['/customers/{customerId}'] + note: > + Source names could be constructed from templates. + An example would be a source name involving a user name or product id. + Although the source name in this case is of high cardinality, + the underlying template is of low cardinality and can be effectively + used for grouping and aggregation. + - id: source.temporary + type: boolean + requirement_level: + recommended: when supported by messaging system and only if the source is temporary. If missing, assumed to be false. + brief: 'A boolean that is true if the message source is temporary and might not exist anymore after messages are processed.' + - id: source.anonymous + type: boolean + requirement_level: + recommended: when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. + brief: 'A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name).' - id: messaging.consumer.synchronous prefix: messaging @@ -148,7 +238,7 @@ groups: brief: > Attributes for RabbitMQ attributes: - - id: routing_key + - id: message.routing_key type: string requirement_level: conditionally_required: If not empty. @@ -163,11 +253,11 @@ groups: brief: > Attributes for Apache Kafka attributes: - - id: message_key + - id: message.key type: string brief: > Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. - They differ from `messaging.message_id` in that they're not unique. + They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. note: > If the key type is not string, it's string representation has to be supplied for the attribute. @@ -184,7 +274,7 @@ groups: brief: > Client Id for the Consumer or Producer that is handling the message. examples: 'client-5' - - id: partition + - id: message.partition type: int brief: > Partition the message is sent to. @@ -194,7 +284,7 @@ groups: brief: > The offset of a record in the corresponding Kafka partition. examples: 42 - - id: tombstone + - id: message.tombstone type: boolean requirement_level: conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. @@ -225,28 +315,28 @@ groups: brief: > The unique identifier for each client. examples: 'myhost@8742@s8083jm' - - id: delivery_timestamp + - id: message.delivery_timestamp type: int requirement_level: conditionally_required: If the message type is delay and delay time level is not specified. brief: > The timestamp in milliseconds that the delay message is expected to be delivered to consumer. examples: 1665987217045 - - id: delay_time_level + - id: message.delay_time_level type: int requirement_level: conditionally_required: If the message type is delay and delivery timestamp is not specified. brief: > The delay time level for delay message, which determines the message delay time. examples: 3 - - id: message_group + - id: message.group type: string requirement_level: conditionally_required: If the message type is FIFO. brief: > It is essential for FIFO message. Messages that belong to the same message group are always processed one by one within the same consumer group. examples: 'myMessageGroup' - - id: message_type + - id: message.type type: allow_custom_values: false members: @@ -264,12 +354,12 @@ groups: brief: 'Transaction message' brief: > Type of message. - - id: message_tag + - id: message.tag type: string brief: > The secondary classifier of message besides topic. examples: tagA - - id: message_keys + - id: message.keys type: string[] brief: > Key(s) of message, another way to mark message besides message id. diff --git a/specification/trace/semantic_conventions/instrumentation/aws-lambda.md b/specification/trace/semantic_conventions/instrumentation/aws-lambda.md index dc4ad1e47d1..c35c23d27b2 100644 --- a/specification/trace/semantic_conventions/instrumentation/aws-lambda.md +++ b/specification/trace/semantic_conventions/instrumentation/aws-lambda.md @@ -123,7 +123,7 @@ added as a link to the span. This means the span may have as many links as messa - [`faas.trigger`][faas]] MUST be set to `pubsub`. - [`messaging.operation`](../messaging.md) MUST be set to `process`. - [`messaging.system`](../messaging.md) MUST be set to `AmazonSQS`. -- [`messaging.destination_kind`](../messaging.md#messaging-attributes) MUST be set to `queue`. +- [`messaging.destination.kind` or `messaging.source.kind`](../messaging.md#messaging-attributes) MUST be set to `queue`. ### SQS Message @@ -136,7 +136,7 @@ added as a link to the span. - [`faas.trigger`][faas] MUST be set to `pubsub`. - [`messaging.operation`](../messaging.md#messaging-attributes) MUST be set to `process`. - [`messaging.system`](../messaging.md#messaging-attributes) MUST be set to `AmazonSQS`. -- [`messaging.destination_kind`](../messaging.md#messaging-attributes) MUST be set to `queue`. +- [`messaging.destination.kind` or `messaging.source.kind`](../messaging.md#messaging-attributes) MUST be set to `queue`. Other [Messaging attributes](../messaging.md#messaging-attributes) SHOULD be set based on the available information in the SQS message event. @@ -219,13 +219,15 @@ Function F: | Span ProcBatch | | SpanKind | `PRODUCER` | `PRODUCER` | `CONSUMER` | `CONSUMER` | `CONSUMER` | | Status | `Ok` | `Ok` | `Ok` | `Ok` | `Ok` | | `messaging.system` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` | `AmazonSQS` | -| `messaging.destination` | `Q` | `Q` | `Q` | `Q` | `Q` | -| `messaging.destination_kind` | `queue` | `queue` | `queue` | `queue` | `queue` | +| `messaging.destination.name` | `Q` | `Q` | | | | +| `messaging.source.name` | | | `Q` | `Q` | `Q` | +| `messaging.destination.kind` | `queue` | `queue` | | | | +| `messaging.source.kind` | | | `queue` | `queue` | `queue` | | `messaging.operation` | | | `process` | `process` | `process` | -| `messaging.message_id` | | | | `"a1"` | `"a2"` | +| `messaging.message.id` | | | | `"a1"` | `"a2"` | Note that if Span Prod1 and Span Prod2 were sent to different queues, Span ProcBatch would not have -`messaging.destination` set as it would correspond to multiple destinations. +`messaging.source.name` set as it would correspond to multiple sources. The above requires user code change to create `Span Proc1` and `Span Proc2`. In Java, the user would inherit from [TracingSqsMessageHandler][] instead of Lambda's standard `RequestHandler` to enable them. Otherwise these two spans diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index faecbe03658..227e173db3a 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -14,13 +14,16 @@ * [Destinations](#destinations) * [Message consumption](#message-consumption) * [Conversations](#conversations) - * [Temporary destinations](#temporary-destinations) + * [Temporary and anonymous destinations](#temporary-and-anonymous-destinations) - [Conventions](#conventions) * [Context propagation](#context-propagation) * [Span name](#span-name) * [Span kind](#span-kind) * [Operation names](#operation-names) - [Messaging attributes](#messaging-attributes) + * [Producer attributes](#producer-attributes) + * [Consumer attributes](#consumer-attributes) + * [Per-message attributes](#per-message-attributes) * [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems) + [RabbitMQ](#rabbitmq) + [Apache Kafka](#apache-kafka) @@ -99,10 +102,10 @@ In some messaging systems, a message can receive one or more reply messages that The grouping usually happens through some sort of "In-Reply-To:" meta information or an explicit *conversation ID* (sometimes called *correlation ID*). Sometimes a conversation can span multiple message destinations (e.g. initiated via a topic, continued on a temporary one-to-one queue). -### Temporary destinations +### Temporary and anonymous destinations Some messaging systems support the concept of *temporary destination* (often only temporary queues) that are established just for a particular set of communication partners (often one to one) or conversation. -Often such destinations are unnamed or have an auto-generated name. +Often such destinations are also *anonymous* - unnamed or have an auto-generated name. ## Conventions @@ -152,20 +155,20 @@ The destination name SHOULD only be used for the span name if it is known to be This can be assumed if it is statically derived from application code or configuration. Wherever possible, the real destination names after resolving logical or aliased names SHOULD be used. If the destination name is dynamic, such as a [conversation ID](#conversations) or a value obtained from a `Reply-To` header, it SHOULD NOT be used for the span name. -In these cases, an artificial destination name that best expresses the destination, or a generic, static fallback like `"(temporary)"` for [temporary destinations](#temporary-destinations) SHOULD be used instead. +In these cases, an artificial destination name that best expresses the destination, or a generic, static fallback like `"(anonymous)"` for [anonymous destinations](#temporary-and-anonymous-destinations) SHOULD be used instead.anonymous The values allowed for `` are defined in the section [Operation names](#operation-names) below. If the format above is used, the operation name MUST match the `messaging.operation` attribute defined for message consumer spans below. Examples: -* `shop.orders send` +* `shop.orders publish` * `shop.orders receive` * `shop.orders process` -* `print_jobs send` +* `print_jobs publish` * `topic with spaces process` * `AuthenticationRequest-Conversations process` -* `(temporary) send` (`(temporary)` being a stable identifier for randomly generated, temporary destination names) +* `(anonymous) publish` (`(anonymous)` being a stable identifier for randomly generated, anonymous destination names) ### Span kind @@ -179,7 +182,7 @@ The following operations related to messages are defined for these semantic conv | Operation name | Description | | -------------- | ----------- | -| `send` | A message is sent to a destination by a message producer/client. | +| `publish` | A message is sent to a destination by a message producer/client. | | `receive` | A message is received from a destination by a message consumer/server. | | `process` | A message that was previously received from a destination is processed by a message consumer/server. | @@ -189,64 +192,136 @@ The following operations related to messages are defined for these semantic conv | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `messaging.system` | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `rocketmq`; `activemq`; `AmazonSQS` | Required | -| `messaging.destination` | string | The message destination name. This might be equal to the span name but is required nevertheless. | `MyQueue`; `MyTopic` | Required | -| `messaging.destination_kind` | string | The kind of message destination | `queue` | Conditionally Required: [1] | -| `messaging.temp_destination` | boolean | A boolean that is true if the message destination is temporary. | | Conditionally Required: [2] | -| `messaging.protocol` | string | The name of the transport protocol. | `AMQP`; `MQTT` | Recommended | -| `messaging.protocol_version` | string | The version of the transport protocol. | `0.9.1` | Recommended | -| `messaging.url` | string | Connection string. | `tibjmsnaming://localhost:7222`; `https://queue.amazonaws.com/80398EXAMPLE/MyQueue` | Recommended | -| `messaging.message_id` | string | A value used by the messaging system as an identifier for the message, represented as a string. | `452a7c7c7c7048c2f887f61572b18fc2` | Recommended | -| `messaging.conversation_id` | string | The [conversation ID](#conversations) identifying the conversation to which the message belongs, represented as a string. Sometimes called "Correlation ID". | `MyConversationId` | Recommended | -| `messaging.message_payload_size_bytes` | int | The (uncompressed) size of the message payload in bytes. Also use this attribute if it is unknown whether the compressed or uncompressed payload size is reported. | `2738` | Recommended | -| `messaging.message_payload_compressed_size_bytes` | int | The compressed size of the message payload in bytes. | `2048` | Recommended | -| [`net.peer.name`](span-general.md) | string | Logical remote hostname, see note below. [3] | `example.com` | Conditionally Required: If available. | -| [`net.sock.family`](span-general.md) | string | Protocol [address family](https://man7.org/linux/man-pages/man7/address_families.7.html) which is used for communication. | `inet6`; `bluetooth` | Conditionally Required: [4] | +| `messaging.operation` | string | A string identifying the kind of messaging operation as defined in the [Operation names](#operation-names) section above. [1] | `publish` | Required | +| `messaging.batch.size` | int | The number of messages sent, received, or processed in the scope of the batching operation. [2] | `0`; `1`; `2` | Conditionally Required: [3] | +| `messaging.message.conversation_id` | string | The [conversation ID](#conversations) identifying the conversation to which the message belongs, represented as a string. Sometimes called "Correlation ID". | `MyConversationId` | Recommended: [4] | +| `messaging.message.id` | string | A value used by the messaging system as an identifier for the message, represented as a string. | `452a7c7c7c7048c2f887f61572b18fc2` | Recommended: [5] | +| `messaging.message.payload_compressed_size_bytes` | int | The compressed size of the message payload in bytes. | `2048` | Recommended: [6] | +| `messaging.message.payload_size_bytes` | int | The (uncompressed) size of the message payload in bytes. Also use this attribute if it is unknown whether the compressed or uncompressed payload size is reported. | `2738` | Recommended: [7] | +| [`net.app.protocol.name`](span-general.md) | string | Application layer protocol used. The value SHOULD be normalized to lowercase. | `amqp`; `mqtt` | Recommended | +| [`net.app.protocol.version`](span-general.md) | string | Version of the application layer protocol used. See note below. [8] | `3.1.1` | Recommended | +| [`net.peer.name`](span-general.md) | string | Logical remote hostname, see note below. [9] | `example.com` | Conditionally Required: If available. | +| [`net.sock.family`](span-general.md) | string | Protocol [address family](https://man7.org/linux/man-pages/man7/address_families.7.html) which is used for communication. | `inet6`; `bluetooth` | Conditionally Required: [10] | | [`net.sock.peer.addr`](span-general.md) | string | Remote socket peer address: IPv4 or IPv6 for internet protocols, path for local communication, [etc](https://man7.org/linux/man-pages/man7/address_families.7.html). | `127.0.0.1`; `/tmp/mysql.sock` | Recommended | -| [`net.sock.peer.name`](span-general.md) | string | Remote socket peer name. | `proxy.example.com` | Recommended: [5] | -| [`net.sock.peer.port`](span-general.md) | int | Remote socket peer port. | `16456` | Recommended: [6] | +| [`net.sock.peer.name`](span-general.md) | string | Remote socket peer name. | `proxy.example.com` | Recommended: [11] | +| [`net.sock.peer.port`](span-general.md) | int | Remote socket peer port. | `16456` | Recommended: [12] | -**[1]:** If the message destination is either a `queue` or `topic`. +**[1]:** If custom value is used, it MUST known to be of low cardinality. -**[2]:** If value is `true`. When missing, the value is assumed to be `false`. +**[2]:** Instrumentations SHOULD NOT set `messaging.batch.size` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.size` for batching APIs and SHOULD NOT use it for single-message APIs. + +**[3]:** If the span describes operation on a batch of messages. + +**[4]:** Only if span represents operation on a single message. + +**[5]:** Only if span represents operation on a single message. + +**[6]:** Only if span represents operation on a single message. + +**[7]:** Only if span represents operation on a single message. -**[3]:** This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from. +**[8]:** `net.app.protocol.version` refers to the version of the protocol used and might be different from the protocol client's version. If the HTTP client used has a version of `0.27.2`, but sends HTTP version `1.1`, this attribute should be set to `1.1`. -**[4]:** If different than `inet` and if any of `net.sock.peer.addr` or `net.sock.host.addr` are set. Consumers of telemetry SHOULD accept both IPv4 and IPv6 formats for the address in `net.sock.peer.addr` if `net.sock.family` is not set. This is to support instrumentations that follow previous versions of this document. +**[9]:** This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from. -**[5]:** If different than `net.peer.name` and if `net.sock.peer.addr` is set. +**[10]:** If different than `inet` and if any of `net.sock.peer.addr` or `net.sock.host.addr` are set. Consumers of telemetry SHOULD accept both IPv4 and IPv6 formats for the address in `net.sock.peer.addr` if `net.sock.family` is not set. This is to support instrumentations that follow previous versions of this document. -**[6]:** If defined for the address family and if different than `net.peer.port` and if `net.sock.peer.addr` is set. +**[11]:** If different than `net.peer.name` and if `net.sock.peer.addr` is set. -`messaging.destination_kind` MUST be one of the following: +**[12]:** If defined for the address family and if different than `net.peer.port` and if `net.sock.peer.addr` is set. + +`messaging.operation` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. | Value | Description | |---|---| -| `queue` | A message sent to a queue | -| `topic` | A message sent to a topic | +| `publish` | publish | +| `receive` | receive | +| `process` | process | Additionally `net.peer.port` from the [network attributes][] is recommended. Furthermore, it is strongly recommended to add the [`net.transport`][] attribute and follow its guidelines, especially for in-process queueing systems (like [Hangfire][], for example). These attributes should be set to the broker to which the message is sent/from which it is received. +Note that attributes in `messaging.message` namespace describe an individual message, `messaging.destination` namespace +contains attributes that describe the logical entity messages are published to, and `messaging.source` describes +logical entity messages are received from. Attributes in `messaging.batch` namespace describe batch properties. + [network attributes]: span-general.md#general-network-connection-attributes [`net.transport`]: span-general.md#network-transport-attributes [Hangfire]: https://www.hangfire.io/ -For message consumers, the following additional attributes may be set: +### Producer attributes + +Following additional attributes describe message producer operations. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `messaging.destination.kind` | string | The kind of message destination | `queue` | Conditionally Required: [1] | +| `messaging.destination.template` | string | Low cardinality field representing messaging destination [2] | `/customers/{customerId}` | Conditionally Required: [3] | +| `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Conditionally Required: [4] | +| `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Conditionally Required: [5] | +| `messaging.destination.name` | string | The message destination name [6] | `MyQueue`; `MyTopic` | Conditionally Required: [7] | + +**[1]:** If the message destination is either a `queue` or `topic`. + +**[2]:** Destination names could be constructed from templates. An example would be a destination name involving a user name or product id. Although the destination name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. + +**[3]:** If available. Instrumentations MUST NOT use `messaging.destination.name` as template unless low-cardinality of destination name is guaranteed. + +**[4]:** If value is `true`. When missing, the value is assumed to be `false`. + +**[5]:** If value is `true`. When missing, the value is assumed to be `false`. + +**[6]:** Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If +broker does not have such notion, destination name SHOULD uniquely identify broker. + +**[7]:** If one message is being published or if the value applies to all messages in the batch. + +`messaging.destination.kind` MUST be one of the following: + +| Value | Description | +|---|---| +| `queue` | A message sent to a queue | +| `topic` | A message sent to a topic | + + +### Consumer attributes + +Following additional attributes describe message consumer operations. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.operation` | string | A string identifying the kind of message consumption as defined in the [Operation names](#operation-names) section above. If the operation is "send", this attribute MUST NOT be set, since the operation can be inferred from the span kind in that case. | `receive` | Recommended | | `messaging.consumer_id` | string | The identifier for the consumer receiving a message. For Kafka, set it to `{messaging.kafka.consumer_group} - {messaging.kafka.client_id}`, if both are present, or only `messaging.kafka.consumer_group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. | `mygroup - client-6` | Recommended | +| `messaging.source.kind` | string | The kind of message source | `queue` | Conditionally Required: [1] | +| `messaging.source.template` | string | Low cardinality field representing messaging source [2] | `/customers/{customerId}` | Conditionally Required: [3] | +| `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [4] | +| `messaging.source.anonymous` | boolean | A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name). | | Recommended: [5] | +| `messaging.source.name` | string | The message source name [6] | `MyQueue`; `MyTopic` | Conditionally Required: [7] | + +**[1]:** If the message source is either a `queue` or `topic`. + +**[2]:** Source names could be constructed from templates. An example would be a source name involving a user name or product id. Although the source name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. + +**[3]:** If available. Instrumentations MUST NOT use `messaging.source.name` as template unless low-cardinality of source name is guaranteed. -`messaging.operation` MUST be one of the following: +**[4]:** when supported by messaging system and only if the source is temporary. If missing, assumed to be false. + +**[5]:** when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. + +**[6]:** Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If +broker does not have such notion, source name SHOULD uniquely identify broker. + +**[7]:** If the value applies to all messages in the batch. + +`messaging.source.kind` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. | Value | Description | |---|---| -| `receive` | receive | -| `process` | process | +| `queue` | A message received from a queue | +| `topic` | A message received from a topic | The *receive* span is be used to track the time used for receiving the message(s), whereas the *process* span(s) track the time for processing the message(s). @@ -256,17 +331,34 @@ For batch receiving and processing (see the [Batch receiving](#batch-receiving) Even though in that case one might think that the processing span's kind should be `INTERNAL`, that kind MUST NOT be used. Instead span kind should be set to either `CONSUMER` or `SERVER` according to the rules defined above. +### Per-message attributes + +All messaging operations (`publish`, `receive`, `process`, or other not covered by this specification) can describe a batch of messages. +Attributes in `messaging.message` or `messaging.{system}.message` namespace describe individual messages. For single-message operations they SHOULD be set on corresponding span. + +For batch operations per-message attributes are usually different and cannot be set on the corresponding span and MAY be set on links. See [Batch Receiving](#batch-receiving) and [Batch Processing](#batch-processing) for more information on correlation using links. + +Some messaging systems such as Kafka, Azure EventGrid allow to publish a single batch of messages to different topics, in such cases attributes in `messaging.destination` and `messaging.source` MAY be +set on links. Corresponding instrumentations MAY set source and destination attributes on the span if all messages in the batch share the same destination or source. + ### Attributes specific to certain messaging systems +All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that are specific to the message, the message destination, the message source, or the whole batch SHOULD be populated under corresponding namespace: + +* message-specific under `messaging.{system}.message` +* destination-specific under `messaging.{system}.destination` +* source-specific under `messaging.{system}.source` +* batch-specific under `messaging.{system}.batch` + #### RabbitMQ In RabbitMQ, the destination is defined by an *exchange* and a *routing key*. -`messaging.destination` MUST be set to the name of the exchange. This will be an empty string if the default exchange is used. +`messaging.destination.name` MUST be set to the name of the exchange. This will be an empty string if the default exchange is used. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.rabbitmq.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | +| `messaging.rabbitmq.message.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | #### Apache Kafka @@ -276,12 +368,12 @@ For Apache Kafka, the following additional attributes are defined: | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.kafka.message_key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message_id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | +| `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | | `messaging.kafka.consumer_group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | | `messaging.kafka.client_id` | string | Client Id for the Consumer or Producer that is handling the message. | `client-5` | Recommended | -| `messaging.kafka.partition` | int | Partition the message is sent to. | `2` | Recommended | +| `messaging.kafka.message.partition` | int | Partition the message is sent to. | `2` | Recommended | | `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | -| `messaging.kafka.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | +| `messaging.kafka.message.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | **[1]:** If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. @@ -302,19 +394,19 @@ Specific attributes for Apache RocketMQ are defined below. | `messaging.rocketmq.namespace` | string | Namespace of RocketMQ resources, resources in different namespaces are individual. | `myNamespace` | Required | | `messaging.rocketmq.client_group` | string | Name of the RocketMQ producer/consumer group that is handling the message. The client type is identified by the SpanKind. | `myConsumerGroup` | Required | | `messaging.rocketmq.client_id` | string | The unique identifier for each client. | `myhost@8742@s8083jm` | Required | -| `messaging.rocketmq.delivery_timestamp` | int | The timestamp in milliseconds that the delay message is expected to be delivered to consumer. | `1665987217045` | Conditionally Required: [1] | -| `messaging.rocketmq.delay_time_level` | int | The delay time level for delay message, which determines the message delay time. | `3` | Conditionally Required: [2] | -| `messaging.rocketmq.message_group` | string | It is essential for FIFO message. Messages that belong to the same message group are always processed one by one within the same consumer group. | `myMessageGroup` | Conditionally Required: If the message type is FIFO. | -| `messaging.rocketmq.message_type` | string | Type of message. | `normal` | Recommended | -| `messaging.rocketmq.message_tag` | string | The secondary classifier of message besides topic. | `tagA` | Recommended | -| `messaging.rocketmq.message_keys` | string[] | Key(s) of message, another way to mark message besides message id. | `[keyA, keyB]` | Recommended | +| `messaging.rocketmq.message.delivery_timestamp` | int | The timestamp in milliseconds that the delay message is expected to be delivered to consumer. | `1665987217045` | Conditionally Required: [1] | +| `messaging.rocketmq.message.delay_time_level` | int | The delay time level for delay message, which determines the message delay time. | `3` | Conditionally Required: [2] | +| `messaging.rocketmq.message.group` | string | It is essential for FIFO message. Messages that belong to the same message group are always processed one by one within the same consumer group. | `myMessageGroup` | Conditionally Required: If the message type is FIFO. | +| `messaging.rocketmq.message.type` | string | Type of message. | `normal` | Recommended | +| `messaging.rocketmq.message.tag` | string | The secondary classifier of message besides topic. | `tagA` | Recommended | +| `messaging.rocketmq.message.keys` | string[] | Key(s) of message, another way to mark message besides message id. | `[keyA, keyB]` | Recommended | | `messaging.rocketmq.consumption_model` | string | Model of message consumption. This only applies to consumer spans. | `clustering` | Recommended | **[1]:** If the message type is delay and delay time level is not specified. **[2]:** If the message type is delay and delivery timestamp is not specified. -`messaging.rocketmq.message_type` MUST be one of the following: +`messaging.rocketmq.message.type` MUST be one of the following: | Value | Description | |---|---| @@ -347,7 +439,7 @@ Process CB: | Span CB1 | | Field or Attribute | Span Prod1 | Span CA1 | Span CB1 | |-|-|-|-| -| Span name | `"T send"` | `"T process"` | `"T process"` | +| Span name | `"T publish"` | `"T process"` | `"T process"` | | Parent | | Span Prod1 | Span Prod1 | | Links | | | | | SpanKind | `PRODUCER` | `CONSUMER` | `CONSUMER` | @@ -355,10 +447,12 @@ Process CB: | Span CB1 | | `net.peer.name` | `"ms"` | `"ms"` | `"ms"` | | `net.peer.port` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination` | `"T"` | `"T"` | `"T"` | -| `messaging.destination_kind` | `"topic"` | `"topic"` | `"topic"` | +| `messaging.destination.name` | `"T"` | | | +| `messaging.destination.kind` | `"topic"` | | | +| `messaging.source.name` | | `"T"` | `"T"` | +| `messaging.source.kind` | | `"topic"` | `"topic"` | | `messaging.operation` | | `"process"` | `"process"` | -| `messaging.message_id` | `"a1"` | `"a1"`| `"a1"` | +| `messaging.message.id` | `"a1"` | `"a1"`| `"a1"` | ### Apache Kafka with Quarkus or Spring Boot Example @@ -382,7 +476,7 @@ Process CB: | Span Rcv2 | | Field or Attribute | Span Prod1 | Span Rcv1 | Span Proc1 | Span Prod2 | Span Rcv2 |-|-|-|-|-|-| -| Span name | `"T1 send"` | `"T1 receive"` | `"T1 process"` | `"T2 send"` | `"T2 receive`" | +| Span name | `"T1 publish"` | `"T1 receive"` | `"T1 process"` | `"T2 publish"` | `"T2 receive`" | | Parent | | Span Prod1 | Span Rcv1 | Span Rcv1 | Span Prod2 | | Links | | | | | | | SpanKind | `PRODUCER` | `CONSUMER` | `CONSUMER` | `PRODUCER` | `CONSUMER` | @@ -390,10 +484,12 @@ Process CB: | Span Rcv2 | | `peer.service` | `"myKafka"` | | | `"myKafka"` | | | `service.name` | | `"myConsumer1"` | `"myConsumer1"` | | `"myConsumer2"` | | `messaging.system` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | -| `messaging.destination` | `"T1"` | `"T1"` | `"T1"` | `"T2"` | `"T2"` | -| `messaging.destination_kind` | `"topic"` | `"topic"` | `"topic"` | `"topic"` | `"topic"` | +| `messaging.destination.name` | `"T1"` | | | | | +| `messaging.destination.kind` | `"topic"` | | | | | +| `messaging.source.name` | | `"T1"` | `"T1"` | `"T2"` | `"T2"` | +| `messaging.source.kind` | | `"topic"` | `"topic"` | `"topic"` | `"topic"` | | `messaging.operation` | | | `"process"` | | `"receive"` | -| `messaging.kafka.message_key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | +| `messaging.kafka.message.key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | | `messaging.kafka.consumer_group` | | `"my-group"` | `"my-group"` | | `"another-group"` | | `messaging.kafka.client_id` | | `"5"` | `"5"` | `"5"` | `"8"` | | `messaging.kafka.partition` | `"1"` | `"1"` | `"1"` | `"3"` | `"3"` | @@ -401,7 +497,7 @@ Process CB: | Span Rcv2 | ### Batch receiving -Given is a process P, that sends two messages to a queue Q on messaging system MS, and a process C, which receives both of them in one batch (Span Recv1) and processes each message separately (Spans Proc1 and Proc2). +Given is a process P, that publishes two messages to a queue Q on messaging system MS, and a process C, which receives both of them in one batch (Span Recv1) and processes each message separately (Spans Proc1 and Proc2). Since a span can only have one parent and the propagated trace and span IDs are not known when the receiving span is started, the receiving span will have no parent and the processing spans are correlated with the producing spans using links. @@ -415,7 +511,7 @@ Process C: | Span Recv1 | | Field or Attribute | Span Prod1 | Span Prod2 | Span Recv1 | Span Proc1 | Span Proc2 | |-|-|-|-|-|-| -| Span name | `"Q send"` | `"Q send"` | `"Q receive"` | `"Q process"` | `"Q process"` | +| Span name | `"Q publish"` | `"Q publish"` | `"Q receive"` | `"Q process"` | `"Q process"` | | Parent | | | | Span Recv1 | Span Recv1 | | Links | | | | Span Prod1 | Span Prod2 | | SpanKind | `PRODUCER` | `PRODUCER` | `CONSUMER` | `CONSUMER` | `CONSUMER` | @@ -423,17 +519,19 @@ Process C: | Span Recv1 | | `net.peer.name` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `net.peer.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | -| `messaging.destination_kind` | `"queue"` | `"queue"` | `"queue"` | `"queue"` | `"queue"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | | | | +| `messaging.destination.kind` | `"queue"` | `"queue"` | | | | +| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | +| `messaging.source.kind` | | | `"queue"` | `"queue"` | `"queue"` | | `messaging.operation` | | | `"receive"` | `"process"` | `"process"` | -| `messaging.message_id` | `"a1"` | `"a2"` | | `"a1"` | `"a2"` | +| `messaging.message.id` | `"a1"` | `"a2"` | | `"a1"` | `"a2"` | +| `messaging.batch.size` | | | 2 | | | ### Batch processing -Given is a process P, that sends two messages to a queue Q on messaging system MS, and a process C, which receives both of them separately (Span Recv1 and Recv2) and processes both messages in one batch (Span Proc1). +Given is a process P, that publishes two messages to a queue Q on messaging system MS, and a process C, which receives them separately in two different operations (Span Recv1 and Recv2) and processes both messages in one batch (Span Proc1). Since each span can only have one parent, C3 should not choose a random parent out of C1 and C2, but rather rely on the implicitly selected parent as defined by the [tracing API spec](../api.md). -Similarly, only one value can be set as `message_id`, so C3 cannot report both `a1` and `a2` and therefore attribute is left out. Depending on the implementation, the producing spans might still be available in the meta data of the messages and should be added to C3 as links. The client library or application could also add the receiver span's SpanContext to the data structure it returns for each message. In this case, C3 could also add links to the receiver spans C1 and C2. @@ -448,15 +546,20 @@ Process C: | Span Recv1 | Span Recv2 | | Field or Attribute | Span Prod1 | Span Prod2 | Span Recv1 | Span Recv2 | Span Proc1 | |-|-|-|-|-|-| -| Span name | `"Q send"` | `"Q send"` | `"Q receive"` | `"Q receive"` | `"Q process"` | +| Span name | `"Q publish"` | `"Q publish"` | `"Q receive"` | `"Q receive"` | `"Q process"` | | Parent | | | Span Prod1 | Span Prod2 | | -| Links | | | | | Span Prod1 + Prod2 | +| Links | | | | | [Span Prod1, Span Prod2 ] | +| Link attributes | | | | | Span Prod1: `messaging.message.id`: `"a1"` | +| | | | | | Span Prod2: `messaging.message.id`: `"a2"` | | SpanKind | `PRODUCER` | `PRODUCER` | `CONSUMER` | `CONSUMER` | `CONSUMER` | | Status | `Ok` | `Ok` | `Ok` | `Ok` | `Ok` | | `net.peer.name` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `net.peer.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | -| `messaging.destination_kind` | `"queue"` | `"queue"` | `"queue"` | `"queue"` | `"queue"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | | | | +| `messaging.destination.kind` | `"queue"` | `"queue"` | | | | +| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | +| `messaging.source.kind` | | | `"queue"` | `"queue"` | `"queue"` | | `messaging.operation` | | | `"receive"` | `"receive"` | `"process"` | -| `messaging.message_id` | `"a1"` | `"a2"` | `"a1"` | `"a2"` | | +| `messaging.message.id` | `"a1"` | `"a2"` | `"a1"` | `"a2"` | | +| `messaging.batch.size` | | | 1 | 1 | 2 | From 81f32852f17318b5b7cf5b80c6be14d551033ce2 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 21 Nov 2022 14:32:27 -0800 Subject: [PATCH 02/20] clarify destination attributes on consumers --- CHANGELOG.md | 4 +-- semantic_conventions/trace/messaging.yaml | 23 ++++++++++++-- .../trace/semantic_conventions/messaging.md | 31 ++++++++++++++----- 3 files changed, 45 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5539805ee82..475e0c69b9d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -142,9 +142,9 @@ release. `messaging.conversation_id` to `messaging.message.conversation_id`, `messaging.message_payload_size_bytes` to `messaging.message.payload_size_bytes`, `messaging.message_payload_compressed_size_bytes` to `messaging.message.payload_compressed_size_bytes`, - `messaging.rabbitmq.routing_key`: `messaging.rabbitmq.message.routing_key`, + `messaging.rabbitmq.routing_key`: `messaging.rabbitmq.destination.routing_key`, `messaging.kafka.message_key` to `messaging.kafka.message.key`, - `messaging.kafka.partition` to `messaging.kafka.message.partition`, + `messaging.kafka.partition` to `messaging.kafka.destination.partition`, `messaging.kafka.tombstone` to `messaging.kafka.message.tombstone`, `messaging.rocketmq.message_type` to `messaging.rocketmq.message.type`, `messaging.rocketmq.message_tag` to `messaging.rocketmq.message.tag`, diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index e7273549544..fbc7b174000 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -66,7 +66,7 @@ groups: brief: > A string identifying the kind of messaging operation as defined in the [Operation names](#operation-names) section above. - note: If custom value is used, it MUST known to be of low cardinality. + note: If a custom value is used, it MUST known to be of low cardinality. - id: batch.size type: int brief: The number of messages sent, received, or processed in the scope of the batching operation. @@ -221,6 +221,18 @@ groups: requirement_level: recommended: when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. brief: 'A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name).' + - ref: messaging.destination.name + requirement_level: + recommended: If known on consumer + - ref: messaging.destination.kind + requirement_level: + recommended: If known on consumer + - ref: messaging.destination.temporary + requirement_level: + recommended: If known on consumer + - ref: messaging.destination.anonymous + requirement_level: + recommended: If known on consumer - id: messaging.consumer.synchronous prefix: messaging @@ -238,7 +250,7 @@ groups: brief: > Attributes for RabbitMQ attributes: - - id: message.routing_key + - id: destination.routing_key type: string requirement_level: conditionally_required: If not empty. @@ -274,7 +286,7 @@ groups: brief: > Client Id for the Consumer or Producer that is handling the message. examples: 'client-5' - - id: message.partition + - id: destination.partition type: int brief: > Partition the message is sent to. @@ -284,6 +296,11 @@ groups: brief: > The offset of a record in the corresponding Kafka partition. examples: 42 + - id: source.partition + type: int + brief: > + Partition the message is received from. + examples: 2 - id: message.tombstone type: boolean requirement_level: diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 227e173db3a..093c9b1cad4 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -155,7 +155,7 @@ The destination name SHOULD only be used for the span name if it is known to be This can be assumed if it is statically derived from application code or configuration. Wherever possible, the real destination names after resolving logical or aliased names SHOULD be used. If the destination name is dynamic, such as a [conversation ID](#conversations) or a value obtained from a `Reply-To` header, it SHOULD NOT be used for the span name. -In these cases, an artificial destination name that best expresses the destination, or a generic, static fallback like `"(anonymous)"` for [anonymous destinations](#temporary-and-anonymous-destinations) SHOULD be used instead.anonymous +In these cases, an artificial destination name that best expresses the destination, or a generic, static fallback like `"(anonymous)"` for [anonymous destinations](#temporary-and-anonymous-destinations) SHOULD be used instead. The values allowed for `` are defined in the section [Operation names](#operation-names) below. If the format above is used, the operation name MUST match the `messaging.operation` attribute defined for message consumer spans below. @@ -206,7 +206,7 @@ The following operations related to messages are defined for these semantic conv | [`net.sock.peer.name`](span-general.md) | string | Remote socket peer name. | `proxy.example.com` | Recommended: [11] | | [`net.sock.peer.port`](span-general.md) | int | Remote socket peer port. | `16456` | Recommended: [12] | -**[1]:** If custom value is used, it MUST known to be of low cardinality. +**[1]:** If a custom value is used, it MUST known to be of low cardinality. **[2]:** Instrumentations SHOULD NOT set `messaging.batch.size` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.size` for batching APIs and SHOULD NOT use it for single-message APIs. @@ -253,7 +253,7 @@ logical entity messages are received from. Attributes in `messaging.batch` names ### Producer attributes -Following additional attributes describe message producer operations. +The following additional attributes describe message producer operations. | Attribute | Type | Description | Examples | Requirement Level | @@ -289,7 +289,7 @@ broker does not have such notion, destination name SHOULD uniquely identify brok ### Consumer attributes -Following additional attributes describe message consumer operations. +The following additional attributes describe message consumer operations. | Attribute | Type | Description | Examples | Requirement Level | @@ -299,7 +299,11 @@ Following additional attributes describe message consumer operations. | `messaging.source.template` | string | Low cardinality field representing messaging source [2] | `/customers/{customerId}` | Conditionally Required: [3] | | `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [4] | | `messaging.source.anonymous` | boolean | A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name). | | Recommended: [5] | -| `messaging.source.name` | string | The message source name [6] | `MyQueue`; `MyTopic` | Conditionally Required: [7] | +| `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Recommended: If known on consumer | +| `messaging.destination.kind` | string | The kind of message destination | `queue` | Recommended: If known on consumer | +| `messaging.destination.name` | string | The message destination name [6] | `MyQueue`; `MyTopic` | Recommended: If known on consumer | +| `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Recommended: If known on consumer | +| `messaging.source.name` | string | The message source name [7] | `MyQueue`; `MyTopic` | Conditionally Required: [8] | **[1]:** If the message source is either a `queue` or `topic`. @@ -311,10 +315,13 @@ Following additional attributes describe message consumer operations. **[5]:** when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. -**[6]:** Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If +**[6]:** Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If +broker does not have such notion, destination name SHOULD uniquely identify broker. + +**[7]:** Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If broker does not have such notion, source name SHOULD uniquely identify broker. -**[7]:** If the value applies to all messages in the batch. +**[8]:** If the value applies to all messages in the batch. `messaging.source.kind` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. @@ -331,6 +338,9 @@ For batch receiving and processing (see the [Batch receiving](#batch-receiving) Even though in that case one might think that the processing span's kind should be `INTERNAL`, that kind MUST NOT be used. Instead span kind should be set to either `CONSUMER` or `SERVER` according to the rules defined above. + +> Note: Consumer spans can have attributes describing source and destination. Since messages could be routed by brokers, source and destination don't always match. If original destination information is available on the consumer, consumer instrumentations SHOULD populate corresponding `messaging.destination` attributes. + ### Per-message attributes All messaging operations (`publish`, `receive`, `process`, or other not covered by this specification) can describe a batch of messages. @@ -358,7 +368,7 @@ In RabbitMQ, the destination is defined by an *exchange* and a *routing key*. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.rabbitmq.message.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | +| `messaging.rabbitmq.destination.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | #### Apache Kafka @@ -371,8 +381,13 @@ For Apache Kafka, the following additional attributes are defined: | `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | | `messaging.kafka.consumer_group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | | `messaging.kafka.client_id` | string | Client Id for the Consumer or Producer that is handling the message. | `client-5` | Recommended | +<<<<<<< HEAD | `messaging.kafka.message.partition` | int | Partition the message is sent to. | `2` | Recommended | | `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | +======= +| `messaging.kafka.destination.partition` | int | Partition the message is sent to. | `2` | Recommended | +| `messaging.kafka.source.partition` | int | Partition the message is received from. | `2` | Recommended | +>>>>>>> dd7251c (clarify destination attributes on consumers) | `messaging.kafka.message.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | **[1]:** If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. From c49c1018a94607ba49cc1aac5fcad53070229376 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 21 Nov 2022 18:15:09 -0800 Subject: [PATCH 03/20] add consumer namespace and describe what should be in destination/source namespaces --- CHANGELOG.md | 4 +++- semantic_conventions/trace/messaging.yaml | 8 ++++---- .../trace/semantic_conventions/messaging.md | 19 ++++++++++--------- 3 files changed, 17 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 475e0c69b9d..807c3671b4e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -133,7 +133,8 @@ release. - Rename `http.retry_count` to `http.resend_count` and clarify its meaning. ([#2743](https://github.com/open-telemetry/opentelemetry-specification/pull/2743)) -- BREAKING: rename `messaging.destination` to `messaging.destination.name`, +- BREAKING: rename `messaging.consumer_id` to `messaging.consumer.id`, + `messaging.destination` to `messaging.destination.name`, `messaging.temp_destination` to `messaging.destination.temporary`, `messaging.destination_kind` to `messaging.destination.kind`, `messaging.message_id` to `messaging.message.id`, @@ -144,6 +145,7 @@ release. `messaging.message_payload_compressed_size_bytes` to `messaging.message.payload_compressed_size_bytes`, `messaging.rabbitmq.routing_key`: `messaging.rabbitmq.destination.routing_key`, `messaging.kafka.message_key` to `messaging.kafka.message.key`, + `messaging.kafka.consumer_group` to `messaging.kafka.consumer.group`, `messaging.kafka.partition` to `messaging.kafka.destination.partition`, `messaging.kafka.tombstone` to `messaging.kafka.message.tombstone`, `messaging.rocketmq.message_type` to `messaging.rocketmq.message.type`, diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index fbc7b174000..48f02969b58 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -173,12 +173,12 @@ groups: span_kind: consumer brief: 'Semantic convention for a consumer of messages received from a messaging system' attributes: - - id: consumer_id + - id: consumer.id type: string brief: > The identifier for the consumer receiving a message. For Kafka, set it to - `{messaging.kafka.consumer_group} - {messaging.kafka.client_id}`, if both are present, or only - `messaging.kafka.consumer_group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` + `{messaging.kafka.consumer.group} - {messaging.kafka.client_id}`, if both are present, or only + `messaging.kafka.consumer.group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. examples: 'mygroup - client-6' - ref: messaging.source.name @@ -275,7 +275,7 @@ groups: If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. examples: 'myKey' - - id: consumer_group + - id: consumer.group type: string brief: > Name of the Kafka Consumer Group that is handling the message. diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 093c9b1cad4..0f3e1640872 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -245,7 +245,8 @@ These attributes should be set to the broker to which the message is sent/from w Note that attributes in `messaging.message` namespace describe an individual message, `messaging.destination` namespace contains attributes that describe the logical entity messages are published to, and `messaging.source` describes -logical entity messages are received from. Attributes in `messaging.batch` namespace describe batch properties. +logical entity messages are received from; attributes in `messaging.batch` namespace describe batch properties and `messaging.consumer` namespace +describes message consumer properties. [network attributes]: span-general.md#general-network-connection-attributes [`net.transport`]: span-general.md#network-transport-attributes @@ -294,7 +295,7 @@ The following additional attributes describe message consumer operations. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.consumer_id` | string | The identifier for the consumer receiving a message. For Kafka, set it to `{messaging.kafka.consumer_group} - {messaging.kafka.client_id}`, if both are present, or only `messaging.kafka.consumer_group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. | `mygroup - client-6` | Recommended | +| `messaging.consumer.id` | string | The identifier for the consumer receiving a message. For Kafka, set it to `{messaging.kafka.consumer.group} - {messaging.kafka.client_id}`, if both are present, or only `messaging.kafka.consumer.group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. | `mygroup - client-6` | Recommended | | `messaging.source.kind` | string | The kind of message source | `queue` | Conditionally Required: [1] | | `messaging.source.template` | string | Low cardinality field representing messaging source [2] | `/customers/{customerId}` | Conditionally Required: [3] | | `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [4] | @@ -353,12 +354,12 @@ set on links. Corresponding instrumentations MAY set source and destination attr ### Attributes specific to certain messaging systems -All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that are specific to the message, the message destination, the message source, or the whole batch SHOULD be populated under corresponding namespace: +All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that describe message, destination, source, consumer, or batch properties SHOULD be populated under corresponding namespace: -* message-specific under `messaging.{system}.message` -* destination-specific under `messaging.{system}.destination` -* source-specific under `messaging.{system}.source` -* batch-specific under `messaging.{system}.batch` +* `messaging.{system}.message` namespace describes individual messages +* `messaging.{system}.destination` namespace describes destination a message (or a batch) are published to, and `messaging.{system}.source` describes source messages were received from. Combination of attributes in these namespaces should uniquely identify the entity and describe it as precisely as possible. For example, if partitioning is supported, attributes should include partition or other important and visible to users entity properties. +* `messaging.{system}.consumer` namespace describes message consumer properties +* `messaging.{system}.batch` namespace describes message batch properties #### RabbitMQ @@ -379,7 +380,7 @@ For Apache Kafka, the following additional attributes are defined: | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | -| `messaging.kafka.consumer_group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | +| `messaging.kafka.consumer.group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | | `messaging.kafka.client_id` | string | Client Id for the Consumer or Producer that is handling the message. | `client-5` | Recommended | <<<<<<< HEAD | `messaging.kafka.message.partition` | int | Partition the message is sent to. | `2` | Recommended | @@ -505,7 +506,7 @@ Process CB: | Span Rcv2 | | `messaging.source.kind` | | `"topic"` | `"topic"` | `"topic"` | `"topic"` | | `messaging.operation` | | | `"process"` | | `"receive"` | | `messaging.kafka.message.key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | -| `messaging.kafka.consumer_group` | | `"my-group"` | `"my-group"` | | `"another-group"` | +| `messaging.kafka.consumer.group` | | `"my-group"` | `"my-group"` | | `"another-group"` | | `messaging.kafka.client_id` | | `"5"` | `"5"` | `"5"` | `"8"` | | `messaging.kafka.partition` | `"1"` | `"1"` | `"1"` | `"3"` | `"3"` | | `messaging.kafka.message.offset` | `"12"` | `"12"` | `"12"` | `"32"` | `"32"` | From eeeaa7fd3ac54689115bd65a15d59a28781b8e0d Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 23 Nov 2022 08:50:24 -0800 Subject: [PATCH 04/20] rename batch.size to batch.message_count --- semantic_conventions/trace/messaging.yaml | 6 +++--- specification/trace/semantic_conventions/messaging.md | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 48f02969b58..6c1bba4b999 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -67,15 +67,15 @@ groups: A string identifying the kind of messaging operation as defined in the [Operation names](#operation-names) section above. note: If a custom value is used, it MUST known to be of low cardinality. - - id: batch.size + - id: batch.message_count type: int brief: The number of messages sent, received, or processed in the scope of the batching operation. requirement_level: conditionally_required: If the span describes operation on a batch of messages. note: > - Instrumentations SHOULD NOT set `messaging.batch.size` on spans that operate with a single message. + Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD - use `messaging.batch.size` for batching APIs and SHOULD NOT use it for single-message APIs. + use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. examples: [0, 1, 2] - ref: messaging.message.id requirement_level: diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 0f3e1640872..cd405e1977e 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -193,7 +193,7 @@ The following operations related to messages are defined for these semantic conv |---|---|---|---|---| | `messaging.system` | string | A string identifying the messaging system. | `kafka`; `rabbitmq`; `rocketmq`; `activemq`; `AmazonSQS` | Required | | `messaging.operation` | string | A string identifying the kind of messaging operation as defined in the [Operation names](#operation-names) section above. [1] | `publish` | Required | -| `messaging.batch.size` | int | The number of messages sent, received, or processed in the scope of the batching operation. [2] | `0`; `1`; `2` | Conditionally Required: [3] | +| `messaging.batch.message_count` | int | The number of messages sent, received, or processed in the scope of the batching operation. [2] | `0`; `1`; `2` | Conditionally Required: [3] | | `messaging.message.conversation_id` | string | The [conversation ID](#conversations) identifying the conversation to which the message belongs, represented as a string. Sometimes called "Correlation ID". | `MyConversationId` | Recommended: [4] | | `messaging.message.id` | string | A value used by the messaging system as an identifier for the message, represented as a string. | `452a7c7c7c7048c2f887f61572b18fc2` | Recommended: [5] | | `messaging.message.payload_compressed_size_bytes` | int | The compressed size of the message payload in bytes. | `2048` | Recommended: [6] | @@ -208,7 +208,7 @@ The following operations related to messages are defined for these semantic conv **[1]:** If a custom value is used, it MUST known to be of low cardinality. -**[2]:** Instrumentations SHOULD NOT set `messaging.batch.size` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.size` for batching APIs and SHOULD NOT use it for single-message APIs. +**[2]:** Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. **[3]:** If the span describes operation on a batch of messages. @@ -541,7 +541,7 @@ Process C: | Span Recv1 | | `messaging.source.kind` | | | `"queue"` | `"queue"` | `"queue"` | | `messaging.operation` | | | `"receive"` | `"process"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | | `"a1"` | `"a2"` | -| `messaging.batch.size` | | | 2 | | | +| `messaging.batch.message_count` | | | 2 | | | ### Batch processing @@ -578,4 +578,4 @@ Process C: | Span Recv1 | Span Recv2 | | `messaging.source.kind` | | | `"queue"` | `"queue"` | `"queue"` | | `messaging.operation` | | | `"receive"` | `"receive"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | `"a1"` | `"a2"` | | -| `messaging.batch.size` | | | 1 | 1 | 2 | +| `messaging.batch.message_count` | | | 1 | 1 | 2 | From 0e48068d4c3e9f9105553bff81aabb0944e45d26 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 30 Nov 2022 12:28:38 -0800 Subject: [PATCH 05/20] lint --- semantic_conventions/trace/messaging.yaml | 4 ++-- specification/trace/semantic_conventions/messaging.md | 1 - 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 6c1bba4b999..2cbd26aa4e1 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -229,10 +229,10 @@ groups: recommended: If known on consumer - ref: messaging.destination.temporary requirement_level: - recommended: If known on consumer + recommended: If known on consumer - ref: messaging.destination.anonymous requirement_level: - recommended: If known on consumer + recommended: If known on consumer - id: messaging.consumer.synchronous prefix: messaging diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index cd405e1977e..c8445c345a4 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -339,7 +339,6 @@ For batch receiving and processing (see the [Batch receiving](#batch-receiving) Even though in that case one might think that the processing span's kind should be `INTERNAL`, that kind MUST NOT be used. Instead span kind should be set to either `CONSUMER` or `SERVER` according to the rules defined above. - > Note: Consumer spans can have attributes describing source and destination. Since messages could be routed by brokers, source and destination don't always match. If original destination information is available on the consumer, consumer instrumentations SHOULD populate corresponding `messaging.destination` attributes. ### Per-message attributes From 10439c5abaabde6b8969475787a600a4d1736dc8 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 30 Nov 2022 12:32:23 -0800 Subject: [PATCH 06/20] Apply suggestions from code review Co-authored-by: Joao Grassi --- semantic_conventions/trace/messaging.yaml | 24 +++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 2cbd26aa4e1..83847854820 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -9,15 +9,15 @@ groups: type: string brief: 'The message destination name' note: | - Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If - broker does not have such notion, destination name SHOULD uniquely identify broker. + Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If + the broker does not have such notion, the destination name SHOULD uniquely identify the broker. examples: ['MyQueue', 'MyTopic'] - id: source.name type: string brief: 'The message source name' note: | - Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If - broker does not have such notion, source name SHOULD uniquely identify broker. + Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If + the broker does not have such notion, the source name SHOULD uniquely identify the broker. examples: ['MyQueue', 'MyTopic'] - id: message.id type: string @@ -44,7 +44,7 @@ groups: prefix: messaging type: span brief: > - This document defines the attributes used in + This document defines general attributes used in messaging systems. attributes: - id: system @@ -66,20 +66,20 @@ groups: brief: > A string identifying the kind of messaging operation as defined in the [Operation names](#operation-names) section above. - note: If a custom value is used, it MUST known to be of low cardinality. + note: If a custom value is used, it MUST be of low cardinality. - id: batch.message_count type: int brief: The number of messages sent, received, or processed in the scope of the batching operation. requirement_level: - conditionally_required: If the span describes operation on a batch of messages. + conditionally_required: If the span describes an operation on a batch of messages. note: > Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. - When client library supports batch and single-message API for the same operation, instrumentations SHOULD + When a messaging client library supports both batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. examples: [0, 1, 2] - ref: messaging.message.id requirement_level: - recommended: Only if span represents operation on a single message. + recommended: Only for spans that represent an operation on a single message. - ref: messaging.message.conversation_id requirement_level: recommended: Only if span represents operation on a single message. @@ -131,7 +131,7 @@ groups: value: "topic" brief: "A message sent to a topic" requirement_level: - conditionally_required: If the message destination is either a `queue` or `topic`. + conditionally_required: If the message destination is either a `queue` or a `topic`. brief: 'The kind of message destination' - id: destination.template type: string @@ -214,12 +214,12 @@ groups: - id: source.temporary type: boolean requirement_level: - recommended: when supported by messaging system and only if the source is temporary. If missing, assumed to be false. + recommended: When supported by messaging system and only if the source is temporary. When missing, the value is assumed to be `false`. brief: 'A boolean that is true if the message source is temporary and might not exist anymore after messages are processed.' - id: source.anonymous type: boolean requirement_level: - recommended: when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. + recommended: When supported by messaging system and only if the source is anonymous. When missing, the value is assumed to be `false`. brief: 'A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name).' - ref: messaging.destination.name requirement_level: From 87f375fc9770cbf5fb0cb120ba44548aaefe09f3 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 30 Nov 2022 12:33:11 -0800 Subject: [PATCH 07/20] update markdown --- .../trace/semantic_conventions/messaging.md | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index c8445c345a4..82b3d5b886d 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -206,15 +206,15 @@ The following operations related to messages are defined for these semantic conv | [`net.sock.peer.name`](span-general.md) | string | Remote socket peer name. | `proxy.example.com` | Recommended: [11] | | [`net.sock.peer.port`](span-general.md) | int | Remote socket peer port. | `16456` | Recommended: [12] | -**[1]:** If a custom value is used, it MUST known to be of low cardinality. +**[1]:** If a custom value is used, it MUST be of low cardinality. -**[2]:** Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. When client library supports batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. +**[2]:** Instrumentations SHOULD NOT set `messaging.batch.message_count` on spans that operate with a single message. When a messaging client library supports both batch and single-message API for the same operation, instrumentations SHOULD use `messaging.batch.message_count` for batching APIs and SHOULD NOT use it for single-message APIs. -**[3]:** If the span describes operation on a batch of messages. +**[3]:** If the span describes an operation on a batch of messages. **[4]:** Only if span represents operation on a single message. -**[5]:** Only if span represents operation on a single message. +**[5]:** Only for spans that represent an operation on a single message. **[6]:** Only if span represents operation on a single message. @@ -265,7 +265,7 @@ The following additional attributes describe message producer operations. | `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Conditionally Required: [5] | | `messaging.destination.name` | string | The message destination name [6] | `MyQueue`; `MyTopic` | Conditionally Required: [7] | -**[1]:** If the message destination is either a `queue` or `topic`. +**[1]:** If the message destination is either a `queue` or a `topic`. **[2]:** Destination names could be constructed from templates. An example would be a destination name involving a user name or product id. Although the destination name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. @@ -275,8 +275,8 @@ The following additional attributes describe message producer operations. **[5]:** If value is `true`. When missing, the value is assumed to be `false`. -**[6]:** Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If -broker does not have such notion, destination name SHOULD uniquely identify broker. +**[6]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If +the broker does not have such notion, the destination name SHOULD uniquely identify the broker. **[7]:** If one message is being published or if the value applies to all messages in the batch. @@ -312,15 +312,15 @@ The following additional attributes describe message consumer operations. **[3]:** If available. Instrumentations MUST NOT use `messaging.source.name` as template unless low-cardinality of source name is guaranteed. -**[4]:** when supported by messaging system and only if the source is temporary. If missing, assumed to be false. +**[4]:** When supported by messaging system and only if the source is temporary. When missing, the value is assumed to be `false`. -**[5]:** when supported by messaging system and only if the source is anonymous. If missing, assumed to be false. +**[5]:** When supported by messaging system and only if the source is anonymous. When missing, the value is assumed to be `false`. -**[6]:** Destination name SHOULD uniquely identify specific queue, topic, or other entity within broker. If -broker does not have such notion, destination name SHOULD uniquely identify broker. +**[6]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If +the broker does not have such notion, the destination name SHOULD uniquely identify the broker. -**[7]:** Source name SHOULD uniquely identify specific queue, topic, or other entity within broker. If -broker does not have such notion, source name SHOULD uniquely identify broker. +**[7]:** Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If +the broker does not have such notion, the source name SHOULD uniquely identify the broker. **[8]:** If the value applies to all messages in the batch. From 5f6ab528928a42d1e0d9df94a2058240a1358cbe Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 30 Nov 2022 15:29:01 -0800 Subject: [PATCH 08/20] refactor messaging.yaml - move destination and source attributes to corresponding groups --- semantic_conventions/trace/messaging.yaml | 161 ++++++++++-------- .../trace/semantic_conventions/messaging.md | 72 ++++---- 2 files changed, 122 insertions(+), 111 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 83847854820..36cd678aede 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -5,20 +5,8 @@ groups: type: span brief: 'Semantic convention describing per-message attributes populated on messaging spans or links.' attributes: - - id: destination.name - type: string - brief: 'The message destination name' - note: | - Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If - the broker does not have such notion, the destination name SHOULD uniquely identify the broker. - examples: ['MyQueue', 'MyTopic'] - - id: source.name - type: string - brief: 'The message source name' - note: | - Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If - the broker does not have such notion, the source name SHOULD uniquely identify the broker. - examples: ['MyQueue', 'MyTopic'] + - ref: messaging.destination.name + - ref: messaging.source.name - id: message.id type: string brief: 'A value used by the messaging system as an identifier for the message, represented as a string.' @@ -40,6 +28,88 @@ groups: brief: 'The compressed size of the message payload in bytes.' examples: 2048 + - id: messaging.destination + prefix: messaging.destination + # todo (https://github.com/open-telemetry/build-tools/issues/123) + type: span + brief: 'Semantic convention for attributes that describe messaging destination on broker' + attributes: + - id: name + type: string + brief: 'The message destination name' + note: | + Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If + the broker does not have such notion, the destination name SHOULD uniquely identify the broker. + examples: ['MyQueue', 'MyTopic'] + - id: kind + type: + allow_custom_values: false + members: + - id: queue + value: "queue" + brief: "A message sent to a queue" + - id: topic + value: "topic" + brief: "A message sent to a topic" + brief: 'The kind of message destination' + - id: template + type: string + brief: Low cardinality field representing messaging destination + note: > + Destination names could be constructed from templates. + An example would be a destination name involving a user name or product id. + Although the destination name in this case is of high cardinality, + the underlying template is of low cardinality and can be effectively + used for grouping and aggregation. + examples: ['/customers/{customerId}'] + - id: temporary + type: boolean + brief: 'A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed.' + - id: anonymous + type: boolean + brief: 'A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name).' + + - id: messaging.source + prefix: messaging.source + # todo (https://github.com/open-telemetry/build-tools/issues/123) + type: span + brief: 'Semantic convention for attributes that describe messaging source on broker' + attributes: + - id: name + type: string + brief: 'The message source name' + note: | + Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If + the broker does not have such notion, the source name SHOULD uniquely identify the broker. + examples: ['MyQueue', 'MyTopic'] + - id: kind + type: + allow_custom_values: true + members: + - id: queue + value: "queue" + brief: "A message received from a queue" + - id: topic + value: "topic" + brief: "A message received from a topic" + brief: 'The kind of message source' + - id: template + type: string + brief: 'Low cardinality field representing messaging source' + examples: ['/customers/{customerId}'] + note: > + Source names could be constructed from templates. + An example would be a source name involving a user name or product id. + Although the source name in this case is of high cardinality, + the underlying template is of low cardinality and can be effectively + used for grouping and aggregation. + - id: temporary + type: boolean + brief: 'A boolean that is true if the message source is temporary and might not exist anymore after messages are processed.' + - id: anonymous + type: boolean + brief: 'A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name).' + - id: messaging prefix: messaging type: span @@ -120,43 +190,21 @@ groups: - ref: messaging.destination.name requirement_level: conditionally_required: If one message is being published or if the value applies to all messages in the batch. - - id: destination.kind - type: - allow_custom_values: false - members: - - id: queue - value: "queue" - brief: "A message sent to a queue" - - id: topic - value: "topic" - brief: "A message sent to a topic" + - ref: messaging.destination.kind requirement_level: conditionally_required: If the message destination is either a `queue` or a `topic`. - brief: 'The kind of message destination' - - id: destination.template - type: string + - ref: messaging.destination.template requirement_level: conditionally_required: > If available. Instrumentations MUST NOT use `messaging.destination.name` as template unless low-cardinality of destination name is guaranteed. - brief: Low cardinality field representing messaging destination - note: > - Destination names could be constructed from templates. - An example would be a destination name involving a user name or product id. - Although the destination name in this case is of high cardinality, - the underlying template is of low cardinality and can be effectively - used for grouping and aggregation. - examples: ['/customers/{customerId}'] - - id: destination.temporary - type: boolean + - ref: messaging.destination.temporary requirement_level: conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. - brief: 'A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed.' - - id: destination.anonymous - type: boolean + - ref: messaging.destination.anonymous requirement_level: conditionally_required: If value is `true`. When missing, the value is assumed to be `false`. - brief: 'A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name).' + - id: messaging.producer.synchronous prefix: messaging type: span @@ -184,43 +232,20 @@ groups: - ref: messaging.source.name requirement_level: conditionally_required: If the value applies to all messages in the batch. - - id: source.kind - type: - allow_custom_values: true - members: - - id: queue - value: "queue" - brief: "A message received from a queue" - - id: topic - value: "topic" - brief: "A message received from a topic" + - ref: messaging.source.kind requirement_level: conditionally_required: If the message source is either a `queue` or `topic`. - brief: 'The kind of message source' - - id: source.template - type: string - brief: 'Low cardinality field representing messaging source' + - ref: messaging.source.template requirement_level: conditionally_required: > If available. Instrumentations MUST NOT use `messaging.source.name` as template unless low-cardinality of source name is guaranteed. - examples: ['/customers/{customerId}'] - note: > - Source names could be constructed from templates. - An example would be a source name involving a user name or product id. - Although the source name in this case is of high cardinality, - the underlying template is of low cardinality and can be effectively - used for grouping and aggregation. - - id: source.temporary - type: boolean + - ref: messaging.source.temporary requirement_level: recommended: When supported by messaging system and only if the source is temporary. When missing, the value is assumed to be `false`. - brief: 'A boolean that is true if the message source is temporary and might not exist anymore after messages are processed.' - - id: source.anonymous - type: boolean + - ref: messaging.source.anonymous requirement_level: recommended: When supported by messaging system and only if the source is anonymous. When missing, the value is assumed to be `false`. - brief: 'A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name).' - ref: messaging.destination.name requirement_level: recommended: If known on consumer diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 82b3d5b886d..47855a520e5 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -259,33 +259,26 @@ The following additional attributes describe message producer operations. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| -| `messaging.destination.kind` | string | The kind of message destination | `queue` | Conditionally Required: [1] | -| `messaging.destination.template` | string | Low cardinality field representing messaging destination [2] | `/customers/{customerId}` | Conditionally Required: [3] | -| `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Conditionally Required: [4] | -| `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Conditionally Required: [5] | -| `messaging.destination.name` | string | The message destination name [6] | `MyQueue`; `MyTopic` | Conditionally Required: [7] | +| `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Conditionally Required: [1] | +| `messaging.destination.kind` | string | The kind of message destination | `queue` | Conditionally Required: [2] | +| `messaging.destination.name` | string | The message destination name [3] | `MyQueue`; `MyTopic` | Conditionally Required: [4] | +| `messaging.destination.template` | string | Low cardinality field representing messaging destination [5] | `/customers/{customerId}` | Conditionally Required: [6] | +| `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Conditionally Required: [7] | -**[1]:** If the message destination is either a `queue` or a `topic`. +**[1]:** If value is `true`. When missing, the value is assumed to be `false`. -**[2]:** Destination names could be constructed from templates. An example would be a destination name involving a user name or product id. Although the destination name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. +**[2]:** If the message destination is either a `queue` or a `topic`. -**[3]:** If available. Instrumentations MUST NOT use `messaging.destination.name` as template unless low-cardinality of destination name is guaranteed. - -**[4]:** If value is `true`. When missing, the value is assumed to be `false`. - -**[5]:** If value is `true`. When missing, the value is assumed to be `false`. - -**[6]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If +**[3]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If the broker does not have such notion, the destination name SHOULD uniquely identify the broker. -**[7]:** If one message is being published or if the value applies to all messages in the batch. +**[4]:** If one message is being published or if the value applies to all messages in the batch. -`messaging.destination.kind` MUST be one of the following: +**[5]:** Destination names could be constructed from templates. An example would be a destination name involving a user name or product id. Although the destination name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. -| Value | Description | -|---|---| -| `queue` | A message sent to a queue | -| `topic` | A message sent to a topic | +**[6]:** If available. Instrumentations MUST NOT use `messaging.destination.name` as template unless low-cardinality of destination name is guaranteed. + +**[7]:** If value is `true`. When missing, the value is assumed to be `false`. ### Consumer attributes @@ -296,40 +289,33 @@ The following additional attributes describe message consumer operations. | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| | `messaging.consumer.id` | string | The identifier for the consumer receiving a message. For Kafka, set it to `{messaging.kafka.consumer.group} - {messaging.kafka.client_id}`, if both are present, or only `messaging.kafka.consumer.group`. For brokers, such as RabbitMQ and Artemis, set it to the `client_id` of the client consuming the message. | `mygroup - client-6` | Recommended | -| `messaging.source.kind` | string | The kind of message source | `queue` | Conditionally Required: [1] | -| `messaging.source.template` | string | Low cardinality field representing messaging source [2] | `/customers/{customerId}` | Conditionally Required: [3] | -| `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [4] | -| `messaging.source.anonymous` | boolean | A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name). | | Recommended: [5] | | `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Recommended: If known on consumer | | `messaging.destination.kind` | string | The kind of message destination | `queue` | Recommended: If known on consumer | -| `messaging.destination.name` | string | The message destination name [6] | `MyQueue`; `MyTopic` | Recommended: If known on consumer | +| `messaging.destination.name` | string | The message destination name [1] | `MyQueue`; `MyTopic` | Recommended: If known on consumer | | `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Recommended: If known on consumer | -| `messaging.source.name` | string | The message source name [7] | `MyQueue`; `MyTopic` | Conditionally Required: [8] | - -**[1]:** If the message source is either a `queue` or `topic`. +| `messaging.source.anonymous` | boolean | A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name). | | Recommended: [2] | +| `messaging.source.kind` | string | The kind of message source | `queue` | Conditionally Required: [3] | +| `messaging.source.name` | string | The message source name [4] | `MyQueue`; `MyTopic` | Conditionally Required: [5] | +| `messaging.source.template` | string | Low cardinality field representing messaging source [6] | `/customers/{customerId}` | Conditionally Required: [7] | +| `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [8] | -**[2]:** Source names could be constructed from templates. An example would be a source name involving a user name or product id. Although the source name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. - -**[3]:** If available. Instrumentations MUST NOT use `messaging.source.name` as template unless low-cardinality of source name is guaranteed. - -**[4]:** When supported by messaging system and only if the source is temporary. When missing, the value is assumed to be `false`. +**[1]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If +the broker does not have such notion, the destination name SHOULD uniquely identify the broker. -**[5]:** When supported by messaging system and only if the source is anonymous. When missing, the value is assumed to be `false`. +**[2]:** When supported by messaging system and only if the source is anonymous. When missing, the value is assumed to be `false`. -**[6]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If -the broker does not have such notion, the destination name SHOULD uniquely identify the broker. +**[3]:** If the message source is either a `queue` or `topic`. -**[7]:** Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If +**[4]:** Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker. If the broker does not have such notion, the source name SHOULD uniquely identify the broker. -**[8]:** If the value applies to all messages in the batch. +**[5]:** If the value applies to all messages in the batch. -`messaging.source.kind` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used. +**[6]:** Source names could be constructed from templates. An example would be a source name involving a user name or product id. Although the source name in this case is of high cardinality, the underlying template is of low cardinality and can be effectively used for grouping and aggregation. -| Value | Description | -|---|---| -| `queue` | A message received from a queue | -| `topic` | A message received from a topic | +**[7]:** If available. Instrumentations MUST NOT use `messaging.source.name` as template unless low-cardinality of source name is guaranteed. + +**[8]:** When supported by messaging system and only if the source is temporary. When missing, the value is assumed to be `false`. The *receive* span is be used to track the time used for receiving the message(s), whereas the *process* span(s) track the time for processing the message(s). From 093d982534639c6356593a6b96e70e607b133c96 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Thu, 1 Dec 2022 12:03:34 -0800 Subject: [PATCH 09/20] Apply suggestions from code review Co-authored-by: Johannes Tax Co-authored-by: Joao Grassi --- .../trace/semantic_conventions/messaging.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 47855a520e5..7900e6e0f0d 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -105,7 +105,7 @@ Sometimes a conversation can span multiple message destinations (e.g. initiated ### Temporary and anonymous destinations Some messaging systems support the concept of *temporary destination* (often only temporary queues) that are established just for a particular set of communication partners (often one to one) or conversation. -Often such destinations are also *anonymous* - unnamed or have an auto-generated name. +Often such destinations are also unnamed (anonymous) or have an auto-generated name. ## Conventions @@ -168,7 +168,7 @@ Examples: * `print_jobs publish` * `topic with spaces process` * `AuthenticationRequest-Conversations process` -* `(anonymous) publish` (`(anonymous)` being a stable identifier for randomly generated, anonymous destination names) +* `(anonymous) publish` (`(anonymous)` being a stable identifier for an unnamed destination) ### Span kind @@ -329,20 +329,20 @@ Instead span kind should be set to either `CONSUMER` or `SERVER` according to th ### Per-message attributes -All messaging operations (`publish`, `receive`, `process`, or other not covered by this specification) can describe a batch of messages. -Attributes in `messaging.message` or `messaging.{system}.message` namespace describe individual messages. For single-message operations they SHOULD be set on corresponding span. +All messaging operations (`publish`, `receive`, `process`, or others not covered by this specification) can describe both single and/or batch of messages. +Attributes in the `messaging.message` or `messaging.{system}.message` namespace describe individual messages. For single-message operations they SHOULD be set on corresponding span. -For batch operations per-message attributes are usually different and cannot be set on the corresponding span and MAY be set on links. See [Batch Receiving](#batch-receiving) and [Batch Processing](#batch-processing) for more information on correlation using links. +For batch operations, per-message attributes are usually different and cannot be set on the corresponding span. In such cases the attributes MAY be set on links. See [Batch Receiving](#batch-receiving) and [Batch Processing](#batch-processing) for more information on correlation using links. -Some messaging systems such as Kafka, Azure EventGrid allow to publish a single batch of messages to different topics, in such cases attributes in `messaging.destination` and `messaging.source` MAY be -set on links. Corresponding instrumentations MAY set source and destination attributes on the span if all messages in the batch share the same destination or source. +Some messaging systems (e.g., Kafka, Azure EventGrid) allow publishing a single batch of messages to different topics. In such cases, the attributes in `messaging.destination` and `messaging.source` MAY be +set on links. Instrumentations MAY set source and destination attributes on the span if all messages in the batch share the same destination or source. ### Attributes specific to certain messaging systems -All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that describe message, destination, source, consumer, or batch properties SHOULD be populated under corresponding namespace: +All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that describe a message, a destination, a source, a consumer or a batch of messages SHOULD be populated under the corresponding namespace: -* `messaging.{system}.message` namespace describes individual messages -* `messaging.{system}.destination` namespace describes destination a message (or a batch) are published to, and `messaging.{system}.source` describes source messages were received from. Combination of attributes in these namespaces should uniquely identify the entity and describe it as precisely as possible. For example, if partitioning is supported, attributes should include partition or other important and visible to users entity properties. +* `messaging.{system}.message`: Describes attributes for individual messages +* `messaging.{system}.destination` and `messaging.{system}.source`: Describes the destination and source a message (or a batch) are published to and received from respectively. The combination of attributes in these namespaces should uniquely identify the entity and describe it as precisely as possible. For example, if partitioning is supported, the attributes should include the partition or other important and visible to users entity properties. * `messaging.{system}.consumer` namespace describes message consumer properties * `messaging.{system}.batch` namespace describes message batch properties From bba1c30b7579cb302400f12cd29e0fc630eca178 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Thu, 1 Dec 2022 12:44:21 -0800 Subject: [PATCH 10/20] more review comments --- semantic_conventions/trace/messaging.yaml | 4 ++-- .../trace/semantic_conventions/messaging.md | 14 ++++++++------ 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 36cd678aede..9f5d86fa526 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -54,7 +54,7 @@ groups: brief: 'The kind of message destination' - id: template type: string - brief: Low cardinality field representing messaging destination + brief: Low cardinality representation of the messaging destination name note: > Destination names could be constructed from templates. An example would be a destination name involving a user name or product id. @@ -95,7 +95,7 @@ groups: brief: 'The kind of message source' - id: template type: string - brief: 'Low cardinality field representing messaging source' + brief: 'Low cardinality representation of the messaging source name' examples: ['/customers/{customerId}'] note: > Source names could be constructed from templates. diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 7900e6e0f0d..32c1541cab4 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -246,7 +246,9 @@ These attributes should be set to the broker to which the message is sent/from w Note that attributes in `messaging.message` namespace describe an individual message, `messaging.destination` namespace contains attributes that describe the logical entity messages are published to, and `messaging.source` describes logical entity messages are received from; attributes in `messaging.batch` namespace describe batch properties and `messaging.consumer` namespace -describes message consumer properties. +describes message consumer properties. +Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace +as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). [network attributes]: span-general.md#general-network-connection-attributes [`net.transport`]: span-general.md#network-transport-attributes @@ -262,7 +264,7 @@ The following additional attributes describe message producer operations. | `messaging.destination.anonymous` | boolean | A boolean that is true if the message destination is anonymous (could be unnamed or have auto-generated name). | | Conditionally Required: [1] | | `messaging.destination.kind` | string | The kind of message destination | `queue` | Conditionally Required: [2] | | `messaging.destination.name` | string | The message destination name [3] | `MyQueue`; `MyTopic` | Conditionally Required: [4] | -| `messaging.destination.template` | string | Low cardinality field representing messaging destination [5] | `/customers/{customerId}` | Conditionally Required: [6] | +| `messaging.destination.template` | string | Low cardinality representation of the messaging destination name [5] | `/customers/{customerId}` | Conditionally Required: [6] | | `messaging.destination.temporary` | boolean | A boolean that is true if the message destination is temporary and might not exist anymore after messages are processed. | | Conditionally Required: [7] | **[1]:** If value is `true`. When missing, the value is assumed to be `false`. @@ -296,7 +298,7 @@ The following additional attributes describe message consumer operations. | `messaging.source.anonymous` | boolean | A boolean that is true if the message source is anonymous (could be unnamed or have auto-generated name). | | Recommended: [2] | | `messaging.source.kind` | string | The kind of message source | `queue` | Conditionally Required: [3] | | `messaging.source.name` | string | The message source name [4] | `MyQueue`; `MyTopic` | Conditionally Required: [5] | -| `messaging.source.template` | string | Low cardinality field representing messaging source [6] | `/customers/{customerId}` | Conditionally Required: [7] | +| `messaging.source.template` | string | Low cardinality representation of the messaging source name [6] | `/customers/{customerId}` | Conditionally Required: [7] | | `messaging.source.temporary` | boolean | A boolean that is true if the message source is temporary and might not exist anymore after messages are processed. | | Recommended: [8] | **[1]:** Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If @@ -342,9 +344,9 @@ set on links. Instrumentations MAY set source and destination attributes on the All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that describe a message, a destination, a source, a consumer or a batch of messages SHOULD be populated under the corresponding namespace: * `messaging.{system}.message`: Describes attributes for individual messages -* `messaging.{system}.destination` and `messaging.{system}.source`: Describes the destination and source a message (or a batch) are published to and received from respectively. The combination of attributes in these namespaces should uniquely identify the entity and describe it as precisely as possible. For example, if partitioning is supported, the attributes should include the partition or other important and visible to users entity properties. -* `messaging.{system}.consumer` namespace describes message consumer properties -* `messaging.{system}.batch` namespace describes message batch properties +* `messaging.{system}.destination` and `messaging.{system}.source`: Describe the destination and source a message (or a batch) are published to and received from respectively. The combination of attributes in these namespaces should uniquely identify the entity and include properties significant for this messaging system. For example, Kafka instrumentations should include partition identifier. +* `messaging.{system}.consumer`: Describes message consumer properties +* `messaging.{system}.batch`: Describes message batch properties #### RabbitMQ From 34c36f3112a6a16bc82993b320b9781d496ebef4 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Thu, 1 Dec 2022 12:50:06 -0800 Subject: [PATCH 11/20] rebase --- semantic_conventions/trace/messaging.yaml | 10 +++++----- specification/trace/semantic_conventions/messaging.md | 10 +++------- 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/semantic_conventions/trace/messaging.yaml b/semantic_conventions/trace/messaging.yaml index 9f5d86fa526..83b049f4561 100644 --- a/semantic_conventions/trace/messaging.yaml +++ b/semantic_conventions/trace/messaging.yaml @@ -316,16 +316,16 @@ groups: brief: > Partition the message is sent to. examples: 2 - - id: message.offset - type: int - brief: > - The offset of a record in the corresponding Kafka partition. - examples: 42 - id: source.partition type: int brief: > Partition the message is received from. examples: 2 + - id: message.offset + type: int + brief: > + The offset of a record in the corresponding Kafka partition. + examples: 42 - id: message.tombstone type: boolean requirement_level: diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 32c1541cab4..b56fdf2ea92 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -246,7 +246,7 @@ These attributes should be set to the broker to which the message is sent/from w Note that attributes in `messaging.message` namespace describe an individual message, `messaging.destination` namespace contains attributes that describe the logical entity messages are published to, and `messaging.source` describes logical entity messages are received from; attributes in `messaging.batch` namespace describe batch properties and `messaging.consumer` namespace -describes message consumer properties. +describes message consumer properties. Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). @@ -344,7 +344,7 @@ set on links. Instrumentations MAY set source and destination attributes on the All attributes that are specific for a messaging system SHOULD be populated in `messaging.{system}` namespace. Attributes that describe a message, a destination, a source, a consumer or a batch of messages SHOULD be populated under the corresponding namespace: * `messaging.{system}.message`: Describes attributes for individual messages -* `messaging.{system}.destination` and `messaging.{system}.source`: Describe the destination and source a message (or a batch) are published to and received from respectively. The combination of attributes in these namespaces should uniquely identify the entity and include properties significant for this messaging system. For example, Kafka instrumentations should include partition identifier. +* `messaging.{system}.destination` and `messaging.{system}.source`: Describe the destination and source a message (or a batch) are published to and received from respectively. The combination of attributes in these namespaces should uniquely identify the entity and include properties significant for this messaging system. For example, Kafka instrumentations should include partition identifier. * `messaging.{system}.consumer`: Describes message consumer properties * `messaging.{system}.batch`: Describes message batch properties @@ -369,13 +369,9 @@ For Apache Kafka, the following additional attributes are defined: | `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | | `messaging.kafka.consumer.group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | | `messaging.kafka.client_id` | string | Client Id for the Consumer or Producer that is handling the message. | `client-5` | Recommended | -<<<<<<< HEAD -| `messaging.kafka.message.partition` | int | Partition the message is sent to. | `2` | Recommended | -| `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | -======= | `messaging.kafka.destination.partition` | int | Partition the message is sent to. | `2` | Recommended | | `messaging.kafka.source.partition` | int | Partition the message is received from. | `2` | Recommended | ->>>>>>> dd7251c (clarify destination attributes on consumers) +| `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | | `messaging.kafka.message.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | **[1]:** If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. From 5e0607236cf6f7bfd979ece28bc63081bc776891 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Fri, 2 Dec 2022 11:18:53 -0800 Subject: [PATCH 12/20] Update specification/trace/semantic_conventions/messaging.md Co-authored-by: Joao Grassi --- specification/trace/semantic_conventions/messaging.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index b56fdf2ea92..9784ed8201b 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -243,7 +243,16 @@ Additionally `net.peer.port` from the [network attributes][] is recommended. Furthermore, it is strongly recommended to add the [`net.transport`][] attribute and follow its guidelines, especially for in-process queueing systems (like [Hangfire][], for example). These attributes should be set to the broker to which the message is sent/from which it is received. -Note that attributes in `messaging.message` namespace describe an individual message, `messaging.destination` namespace +## Attribute namespaces + +- `messaging.message`: Contains attributes that describe individual messages +- `messaging.destination`: Contains attributes that describe the logical entity messages are published to +- `messaging.source`: Contains attributes that describe the logical entity messages are received from +- `messaging.batch`: Contains attributes that describe batch operations +- `messaging.consumer`: Contains attributes that describe message consumers + +Note: Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace +as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). contains attributes that describe the logical entity messages are published to, and `messaging.source` describes logical entity messages are received from; attributes in `messaging.batch` namespace describe batch properties and `messaging.consumer` namespace describes message consumer properties. From 9e2d2a5dc8f4b008980defaec2fd6c6b4c91b4fe Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 5 Dec 2022 10:50:42 -0800 Subject: [PATCH 13/20] review and lint --- specification/trace/semantic_conventions/messaging.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 9784ed8201b..bc84820d407 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -21,6 +21,7 @@ * [Span kind](#span-kind) * [Operation names](#operation-names) - [Messaging attributes](#messaging-attributes) + * [Attribute namespaces](#attribute-namespaces) * [Producer attributes](#producer-attributes) * [Consumer attributes](#consumer-attributes) * [Per-message attributes](#per-message-attributes) @@ -243,7 +244,7 @@ Additionally `net.peer.port` from the [network attributes][] is recommended. Furthermore, it is strongly recommended to add the [`net.transport`][] attribute and follow its guidelines, especially for in-process queueing systems (like [Hangfire][], for example). These attributes should be set to the broker to which the message is sent/from which it is received. -## Attribute namespaces +### Attribute namespaces - `messaging.message`: Contains attributes that describe individual messages - `messaging.destination`: Contains attributes that describe the logical entity messages are published to @@ -251,11 +252,6 @@ These attributes should be set to the broker to which the message is sent/from w - `messaging.batch`: Contains attributes that describe batch operations - `messaging.consumer`: Contains attributes that describe message consumers -Note: Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace -as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). -contains attributes that describe the logical entity messages are published to, and `messaging.source` describes -logical entity messages are received from; attributes in `messaging.batch` namespace describe batch properties and `messaging.consumer` namespace -describes message consumer properties. Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). From 1dbe0d4f620bce79c34bf1635ddf646b545e1372 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 14 Dec 2022 10:42:18 -0800 Subject: [PATCH 14/20] rebase --- schemas/1.16.0 | 35 ----------------------------------- 1 file changed, 35 deletions(-) delete mode 100644 schemas/1.16.0 diff --git a/schemas/1.16.0 b/schemas/1.16.0 deleted file mode 100644 index 9e1381d9e73..00000000000 --- a/schemas/1.16.0 +++ /dev/null @@ -1,35 +0,0 @@ -file_format: 1.1.0 -schema_url: https://opentelemetry.io/schemas/1.16.0 -versions: - 1.16.0: - 1.15.0: - spans: - changes: - # https://github.com/open-telemetry/opentelemetry-specification/pull/2743 - - rename_attributes: - attribute_map: - http.retry_count: http.resend_count - 1.14.0: - 1.13.0: - spans: - changes: - # https://github.com/open-telemetry/opentelemetry-specification/pull/2614 - - rename_attributes: - attribute_map: - net.peer.ip: net.sock.peer.addr - net.host.ip: net.sock.host.addr - 1.12.0: - 1.11.0: - 1.10.0: - 1.9.0: - 1.8.0: - spans: - changes: - - rename_attributes: - attribute_map: - db.cassandra.keyspace: db.name - db.hbase.namespace: db.name - 1.7.0: - 1.6.1: - 1.5.0: - 1.4.0: From 4ba5b63abf6f7f6e43f2c0ce41dc43a7d16b1c8a Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 14 Dec 2022 11:16:41 -0800 Subject: [PATCH 15/20] review --- schemas/1.16.0 | 35 +++++++++++++++++++ .../trace/semantic_conventions/messaging.md | 4 +-- 2 files changed, 37 insertions(+), 2 deletions(-) create mode 100644 schemas/1.16.0 diff --git a/schemas/1.16.0 b/schemas/1.16.0 new file mode 100644 index 00000000000..9e1381d9e73 --- /dev/null +++ b/schemas/1.16.0 @@ -0,0 +1,35 @@ +file_format: 1.1.0 +schema_url: https://opentelemetry.io/schemas/1.16.0 +versions: + 1.16.0: + 1.15.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2743 + - rename_attributes: + attribute_map: + http.retry_count: http.resend_count + 1.14.0: + 1.13.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2614 + - rename_attributes: + attribute_map: + net.peer.ip: net.sock.peer.addr + net.host.ip: net.sock.host.addr + 1.12.0: + 1.11.0: + 1.10.0: + 1.9.0: + 1.8.0: + spans: + changes: + - rename_attributes: + attribute_map: + db.cassandra.keyspace: db.name + db.hbase.namespace: db.name + 1.7.0: + 1.6.1: + 1.5.0: + 1.4.0: diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index bc84820d407..197d969fa4a 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -292,6 +292,8 @@ the broker does not have such notion, the destination name SHOULD uniquely ident The following additional attributes describe message consumer operations. +> Note: Consumer spans can have attributes describing source and destination. Since messages could be routed by brokers, source and destination don't always match. If original destination information is available on the consumer, consumer instrumentations SHOULD populate corresponding `messaging.destination` attributes. + | Attribute | Type | Description | Examples | Requirement Level | |---|---|---|---|---| @@ -332,8 +334,6 @@ For batch receiving and processing (see the [Batch receiving](#batch-receiving) Even though in that case one might think that the processing span's kind should be `INTERNAL`, that kind MUST NOT be used. Instead span kind should be set to either `CONSUMER` or `SERVER` according to the rules defined above. -> Note: Consumer spans can have attributes describing source and destination. Since messages could be routed by brokers, source and destination don't always match. If original destination information is available on the consumer, consumer instrumentations SHOULD populate corresponding `messaging.destination` attributes. - ### Per-message attributes All messaging operations (`publish`, `receive`, `process`, or others not covered by this specification) can describe both single and/or batch of messages. From fb7914bbdf04af3b7aa8c2b8cfae28c2c85a5999 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 19 Dec 2022 09:55:09 -0800 Subject: [PATCH 16/20] add missing schema file --- schemas/1.17.0 | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 schemas/1.17.0 diff --git a/schemas/1.17.0 b/schemas/1.17.0 new file mode 100644 index 00000000000..98808e60800 --- /dev/null +++ b/schemas/1.17.0 @@ -0,0 +1,59 @@ +file_format: 1.1.0 +schema_url: https://opentelemetry.io/schemas/1.16.0 +versions: + 1.17.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2763 + - rename_attributes: + attribute_map: + messaging.consumer_id: messaging.consumer.id + messaging.protocol: net.app.protocol.name + messaging.protocol_version: net.app.protocol.version + messaging.destination: messaging.destination.name + messaging.temp_destination: messaging.destination.temporary + messaging.destination_kind: messaging.destination.kind + messaging.message_id: messaging.message.id + messaging.conversation_id: messaging.message.conversation_id + messaging.message_payload_size_bytes: messaging.message.payload_size_bytes + messaging.message_payload_compressed_size_bytes: messaging.message.payload_compressed_size_bytes + messaging.rabbitmq.routing_key: messaging.rabbitmq.destination.routing_key + messaging.kafka.message_key: messaging.kafka.message.key + messaging.kafka.partition: messaging.kafka.destination.partition + messaging.kafka.tombstone: messaging.kafka.message.tombstone + messaging.rocketmq.message_type: messaging.rocketmq.message.type + messaging.rocketmq.message_tag: messaging.rocketmq.message.tag + messaging.rocketmq.message_keys: messaging.rocketmq.message.keys + messaging.kafka.consumer_group: messaging.kafka.consumer.group + 1.16.0: + 1.15.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2743 + - rename_attributes: + attribute_map: + http.retry_count: http.resend_count + 1.14.0: + 1.13.0: + spans: + changes: + # https://github.com/open-telemetry/opentelemetry-specification/pull/2614 + - rename_attributes: + attribute_map: + net.peer.ip: net.sock.peer.addr + net.host.ip: net.sock.host.addr + 1.12.0: + 1.11.0: + 1.10.0: + 1.9.0: + 1.8.0: + spans: + changes: + - rename_attributes: + attribute_map: + db.cassandra.keyspace: db.name + db.hbase.namespace: db.name + 1.7.0: + 1.6.1: + 1.5.0: + 1.4.0: From 70e04817d5c62a26dfef9a070a39d4c1cb1e0b32 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 19 Dec 2022 10:29:36 -0800 Subject: [PATCH 17/20] add some notes on namespaces and terminology --- .../trace/semantic_conventions/messaging.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 197d969fa4a..ff26b1e848a 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -75,12 +75,15 @@ process of notifying an intermediary that a message was processed successfully. An "intermediary" receives a message to forward it to the next receiver, which might be another intermediary or a consumer. -### Destinations +### Destinations and sources A destination is usually identified by some name unique within the messaging system instance, which might look like a URL or a simple one-word identifier. Traditional messaging, such as JMS, involves two kinds of destinations: *topic*s and *queue*s. A message that is sent (the send-operation is often called "*publish*" in this context) to a *topic* is broadcasted to all consumers that have *subscribed* to the topic. -A message submitted to a queue is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). + +A message submitted to a queue is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). + +Entity messages are received from is called **source** here. Messages can be routed within one or multiple brokers, so message's *source* and *destination* could be different. In a messaging system such as Apache Kafka, all destinations are *topic*s. Each record, or message, is sent to a single consumer per consumer group. @@ -247,10 +250,13 @@ These attributes should be set to the broker to which the message is sent/from w ### Attribute namespaces - `messaging.message`: Contains attributes that describe individual messages -- `messaging.destination`: Contains attributes that describe the logical entity messages are published to +- `messaging.destination`: Contains attributes that describe the logical entity messages are published to. + See [Destinations and sources](#destinations-and-sources) for more details - `messaging.source`: Contains attributes that describe the logical entity messages are received from - `messaging.batch`: Contains attributes that describe batch operations -- `messaging.consumer`: Contains attributes that describe message consumers +- `messaging.consumer`: Contains attributes that describe application instance that consumes a message. See [consumer](#consumer) for more details + +Communication with broker is described with general [network attributes]. Messaging system-specific attributes MUST be defined in the corresponding `messaging.{system}` namespace as described in [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems). From 7f0774cc2630e09a03a787a0950a021e41458111 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Mon, 19 Dec 2022 10:41:02 -0800 Subject: [PATCH 18/20] lint --- schemas/1.17.0 | 2 +- specification/trace/semantic_conventions/messaging.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/schemas/1.17.0 b/schemas/1.17.0 index 98808e60800..46e3180f912 100644 --- a/schemas/1.17.0 +++ b/schemas/1.17.0 @@ -1,5 +1,5 @@ file_format: 1.1.0 -schema_url: https://opentelemetry.io/schemas/1.16.0 +schema_url: https://opentelemetry.io/schemas/1.17.0 versions: 1.17.0: spans: diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index ff26b1e848a..3bc24168ecf 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -11,7 +11,7 @@ * [Producer](#producer) * [Consumer](#consumer) * [Intermediary](#intermediary) - * [Destinations](#destinations) + * [Destinations and sources](#destinations-and-sources) * [Message consumption](#message-consumption) * [Conversations](#conversations) * [Temporary and anonymous destinations](#temporary-and-anonymous-destinations) @@ -81,7 +81,7 @@ A destination is usually identified by some name unique within the messaging sys Traditional messaging, such as JMS, involves two kinds of destinations: *topic*s and *queue*s. A message that is sent (the send-operation is often called "*publish*" in this context) to a *topic* is broadcasted to all consumers that have *subscribed* to the topic. -A message submitted to a queue is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). +A message submitted to a queue is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). Entity messages are received from is called **source** here. Messages can be routed within one or multiple brokers, so message's *source* and *destination* could be different. From 1966d8423ea831bb2480135b56a6d00b46d060d8 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 21 Dec 2022 14:23:08 -0800 Subject: [PATCH 19/20] Apply suggestions from code review Co-authored-by: Joao Grassi --- specification/trace/semantic_conventions/messaging.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 3bc24168ecf..81dd23dff7a 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -77,11 +77,11 @@ might be another intermediary or a consumer. ### Destinations and sources -A destination is usually identified by some name unique within the messaging system instance, which might look like a URL or a simple one-word identifier. +A destination is usually uniquely identified by name within the messaging system instance. Examples of a destination name would be a URL or a simple one-word identifier. Traditional messaging, such as JMS, involves two kinds of destinations: *topic*s and *queue*s. A message that is sent (the send-operation is often called "*publish*" in this context) to a *topic* is broadcasted to all consumers that have *subscribed* to the topic. -A message submitted to a queue is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). +A message that is sent to a *queue* is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). Entity messages are received from is called **source** here. Messages can be routed within one or multiple brokers, so message's *source* and *destination* could be different. From 084d6dbfb095a62a393e9d4b252ca157ebd23c68 Mon Sep 17 00:00:00 2001 From: Liudmila Molkova Date: Wed, 21 Dec 2022 18:32:25 -0800 Subject: [PATCH 20/20] Review comments --- specification/trace/semantic_conventions/messaging.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/trace/semantic_conventions/messaging.md index 81dd23dff7a..9b23b62a3f0 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/trace/semantic_conventions/messaging.md @@ -79,11 +79,12 @@ might be another intermediary or a consumer. A destination is usually uniquely identified by name within the messaging system instance. Examples of a destination name would be a URL or a simple one-word identifier. Traditional messaging, such as JMS, involves two kinds of destinations: *topic*s and *queue*s. + A message that is sent (the send-operation is often called "*publish*" in this context) to a *topic* is broadcasted to all consumers that have *subscribed* to the topic. A message that is sent to a *queue* is processed by a message *consumer* (usually exactly once although some message systems support a more performant at-least-once mode for messages with [idempotent][] processing). -Entity messages are received from is called **source** here. Messages can be routed within one or multiple brokers, so message's *source* and *destination* could be different. +A source represents an entity within messaging system messages are consumed from. Source and destination for specific message may be the same. However, if message is routed within one or multiple brokers, source and destination can be different. In a messaging system such as Apache Kafka, all destinations are *topic*s. Each record, or message, is sent to a single consumer per consumer group.