Skip to content

Commit

Permalink
Merge pull request #2517 from MicrosoftDocs/main638726473635850547syn…
Browse files Browse the repository at this point in the history
…c_temp

Repo sync for protected branch
  • Loading branch information
Jak-MS authored Jan 16, 2025
2 parents 98a6f6b + f3994ef commit 4ab9dc2
Show file tree
Hide file tree
Showing 17 changed files with 699 additions and 13 deletions.
2 changes: 2 additions & 0 deletions data-explorer/kusto-tocs/management/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ items:
items:
- name: Best practices for schema management
href: /kusto/management/management-best-practices?view=azure-data-explorer&preserve-view=true
- name: Avrotize k2a tool
href: /kusto/management/avrotize?view=azure-data-explorer&preserve-view=true
- name: Clusters
items:
- name: .execute cluster script command
Expand Down
28 changes: 18 additions & 10 deletions data-explorer/kusto/management/avro-mapping.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: AVRO Mapping
description: Learn how to use AVRO mapping to map data to columns inside tables upon ingestion.
ms.topic: reference
ms.date: 08/11/2024
ms.date: 01/12/2025
---

# AVRO mapping
Expand All @@ -15,12 +15,12 @@ Use AVRO mapping to map incoming data to columns inside tables when your ingesti

Each AVRO mapping element must contain either of the following optional properties:

| Property | Type | Description |
|------------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Field | `string` | Name of the field in the AVRO record. |
| Path | `string` | If the value starts with `$` it's interpreted as the path to the field in the AVRO document that will become the content of the column in the table. The path that denotes the entire AVRO record is `$`. If the value doesn't start with `$` it's interpreted as a constant value. Paths that include special characters should be escaped as [\'Property Name\']. For more information, see [JSONPath syntax](../query/jsonpath.md). |
| ConstValue | `string` | The constant value to be used for a column instead of some value inside the AVRO file. |
| Transform | `string` | Transformation that should be applied on the content with [mapping transformations](mappings.md#mapping-transformations). |
| Property | Type | Description |
|--|--|--|
| Field | `string` | Name of the field in the AVRO record. |
| Path | `string` | If the value starts with `$`, it's treated as the path to the field in the AVRO document. This path specifies the part of the AVRO document that becomes the content of the column in the table. The path that denotes the entire AVRO record is `$`. If the value doesn't start with `$`, it's treated as a constant value. Paths that include special characters should be escaped as [\'Property Name\']. For more information, see [JSONPath syntax](../query/jsonpath.md). |
| ConstValue | `string` | The constant value to be used for a column instead of some value inside the AVRO file. |
| Transform | `string` | Transformation that should be applied on the content with [mapping transformations](mappings.md#mapping-transformations). |

>[!NOTE]
>
Expand All @@ -44,6 +44,10 @@ Each AVRO mapping element must contain either of the following optional properti
## Examples
### JSON serialization
The following example mapping is serialized as a JSON string when provided as part of the `.ingest` management command.
``` json
[
{"Column": "event_timestamp", "Properties": {"Field": "Timestamp"}},
Expand All @@ -55,7 +59,7 @@ Each AVRO mapping element must contain either of the following optional properti
]
```
The mapping above is serialized as a JSON string when it's provided as part of the `.ingest` management command.
Here the serialized JSON mapping is included in the context of the `.ingest` management command.

````kusto
.ingest into Table123 (@"source1", @"source2")
Expand All @@ -72,9 +76,9 @@ The mapping above is serialized as a JSON string when it's provided as part of t
)
````

### Pre-created mapping
### Precreated mapping

When the mapping is [pre-created](create-ingestion-mapping-command.md), reference the mapping by name in the `.ingest` management command.
When the mapping is [precreated](create-ingestion-mapping-command.md), reference the mapping by name in the `.ingest` management command.

```kusto
.ingest into Table123 (@"source1", @"source2")
Expand All @@ -96,3 +100,7 @@ Use AVRO mapping during ingestion without defining a mapping schema (see [identi
format="AVRO"
)
```

## Related content

* Use the [avrotize k2a](../tools/avrotize.md) tool to create an Avro schema.
2 changes: 2 additions & 0 deletions data-explorer/kusto/management/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ items:
items:
- name: Best practices for schema management
href: management-best-practices.md
- name: Avrotize k2a tool
href: /kusto/management/tools/avrotize.md
- name: Clusters
items:
- name: .execute cluster script command
Expand Down
5 changes: 2 additions & 3 deletions data-explorer/kusto/query/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ ms.topic: reference
ms.custom: build-2023, build-2023-dataai
ms.date: 08/11/2024
adobe-target: true
monikerRange: "microsoft-fabric || azure-data-explorer || microsoft-sentinel || azure-monitor "
monikerRange: "microsoft-fabric || azure-data-explorer || azure-monitor"
---
# Kusto Query Language (KQL) overview

> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)]
> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)]

Kusto Query Language (KQL) is a powerful tool to explore your data and discover patterns, identify anomalies and outliers, create statistical modeling, and more.
Expand Down Expand Up @@ -91,7 +91,6 @@ For more information on management commands, see [Management commands overview](
KQL is used by many other Microsoft services. For specific information on the use of KQL in these environments, refer to the following links:

* [Log queries in Azure Monitor](/azure/azure-monitor/logs/log-query-overview)
* [Kusto Query Language in Microsoft Sentinel](/azure/sentinel/kusto-overview)
* [Understanding the Azure Resource Graph query language](/azure/governance/resource-graph/concepts/query-language)
* [Proactively hunt for threats with advanced hunting in Microsoft 365 Defender](/microsoft-365/security/defender/advanced-hunting-overview)
* [CMPivot queries](/mem/configmgr/core/servers/manage/cmpivot-overview#queries)
Expand Down
Loading

0 comments on commit 4ab9dc2

Please sign in to comment.