Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repo sync for protected branch #2500

Merged
merged 32 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
8a1040d
added permisions description for URI blob import
EdB-MSFT Dec 4, 2024
da89d93
restet .vscode/settings.json
EdB-MSFT Dec 4, 2024
d507980
fix
EdB-MSFT Dec 4, 2024
228d87e
one-time ingest
EdB-MSFT Dec 4, 2024
6c442b2
link to kusto/api/connection-strings/generate-sas-token
EdB-MSFT Dec 4, 2024
63b055b
Update show-tables-command.md
EdB-MSFT Dec 9, 2024
78f3e60
Update ms.date in show-tables-command.md
EdB-MSFT Dec 9, 2024
2e32585
updates
EdB-MSFT Dec 16, 2024
89b899c
fixes
EdB-MSFT Dec 16, 2024
739644d
fixes2
EdB-MSFT Dec 16, 2024
13892e2
fixes3
EdB-MSFT Dec 16, 2024
9e036a6
fixes3
EdB-MSFT Dec 16, 2024
0ad3a11
updated generate sas token
EdB-MSFT Dec 19, 2024
28a0388
updated facet
EdB-MSFT Dec 19, 2024
3516803
Add examples and sample data table
EdB-MSFT Dec 19, 2024
07ba35a
Remove redundant header from output example
EdB-MSFT Dec 19, 2024
f5c784b
added run query button
EdB-MSFT Dec 19, 2024
9888c6a
fix
EdB-MSFT Dec 19, 2024
2759b6f
Delete [!div
EdB-MSFT Dec 19, 2024
fa8b9d8
added note on keywords
EdB-MSFT Dec 22, 2024
3813036
fix
EdB-MSFT Dec 22, 2024
faa5276
Update data-explorer/kusto/api/connection-strings/generate-sas-token.md
shsagir Jan 6, 2025
e73eba1
Update data-explorer/kusto/management/show-tables-command.md
shsagir Jan 6, 2025
a4430de
Update data-explorer/kusto/management/show-tables-command.md
shsagir Jan 6, 2025
4dc7b02
Update data-explorer/kusto/query/index.md
shsagir Jan 6, 2025
59192b9
Merge pull request #6486 from EdB-MSFT/reserved-words
v-dirichards Jan 6, 2025
64cb534
fixing spacing
v-dirichards Jan 6, 2025
6d6df33
Merge pull request #6445 from EdB-MSFT/uuf-permissions-storege-ingest
v-dirichards Jan 6, 2025
baedcde
Merge pull request #6484 from EdB-MSFT/uuf-update-facet
v-dirichards Jan 6, 2025
4809de0
Merge pull request #6461 from EdB-MSFT/patch-1
v-dirichards Jan 6, 2025
831e042
Merge pull request #6501 from MicrosoftDocs/main
Taojunshen Jan 6, 2025
dfe31e6
Merging changes synced from https://github.com/MicrosoftDocs/dataexpl…
Jan 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions data-explorer/get-data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ title: Get data from Azure storage
description: Learn how to get data from Azure storage in Azure Data Explorer.
ms.reviewer: sharmaanshul
ms.topic: how-to
ms.date: 12/04/2023
ms.date: 12/04/2024
---

# Get data from Azure storage

Data ingestion is the process used to load data from one or more sources into a table in Azure Data Explorer. Once ingested, the data becomes available for query. In this article, you learn how to get data from Azure storage (ADLS Gen2 container, blob container, or individual blobs) into either a new or existing table.

Ingestion can be done as a one-time operation, or as a continuous method. Continuous ingestion can be configured via portal only.
Ingestion from an Azure storage account is a one-time operation. To ingest data continuously, see [Configure streaming ingestion](ingest-data-streaming.md).

For general information on data ingestion, see [Azure Data Explorer data ingestion overview](ingest-data-overview.md).

Expand Down Expand Up @@ -59,14 +59,16 @@ Select the data source from the available list. In this example, you're ingestin
| Folder path| Filters data to ingest files with a specific folder path. |
| File extension| Filters data to ingest files with a specific file extension only.|

1. If you selected **Add URI**, paste your storage connection string for a blob container or individual files in the **URI** field, and then select **+**.
1. If you selected **Add URI**, from the storage account, generate an SAS URL for the container or individual blobs you want to ingest. Set the permissions to **Read** and **List** for containers or **Read** for individual blobs. For more information, see [Generate a SAS token](/kusto/api/connection-strings/generate-sas-token?view=azure-data-explorer&preserve-view=true).

1. Paste the URL into the **URI** field, and then select plus (**+**). You can add multiple URIs for individual blobs, or a single URI for a container.

:::image type="content" source="media/get-data-storage/add-uri.png" alt-text="Screenshot of configure tab with the connection string pasted in the URI field." lightbox="media/get-data-storage/add-uri.png":::

> [!NOTE]
>
> * You can add up to 10 individual blobs. Each blob can be a max of 1 GB uncompressed.
> * You can ingest up to 5000 blobs from a single container.

:::image type="content" source="media/get-data-storage/add-uri.png" alt-text="Screenshot of configure tab with the connection string pasted in the URI field." lightbox="media/get-data-storage/add-uri.png":::
> * You can't ingest individual blobs and containers in the same ingestion.

1. Select **Next**

Expand Down
16 changes: 8 additions & 8 deletions data-explorer/kusto/api/connection-strings/generate-sas-token.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,27 +9,27 @@ ms.date: 08/11/2024

> [!INCLUDE [applies](../../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../../includes/applies-to-version/azure-data-explorer.md)]

This article shows you how to generate a SAS token to a resource.
This article shows you how to generate a SAS token to a storage resource. You can generate SAS tokens for containers and individual blobs.

## Azure portal

1. In the [Azure portal](https://portal.azure.com/), open **Storage accounts**.
1. Select the storage account that contains the resource for which you'd like to create a SAS token.
1. From the left menu, select **Containers**.
1. Right-click on the container for which you'd like to create a SAS token.
1. From the context menu, select **Generate SAS**.
1. Navigate to the container, or drill down to the individual blob for which you'd like to create a SAS token.
1. Right-click on the container or blob and select **Generate SAS** from the context menu.

:::image type="content" source="storage/generate-sas-storage-account.png" alt-text="Screenshot of Azure portal with Containers selected. Specific container is right-clicked and a menu opens. Generate SAS is selected from this menu.":::
:::image type="content" source="storage/generate-sas-storage-account.png" lightbox="storage/generate-sas-storage-account.png" alt-text="Screenshot of Azure portal with Containers selected. Specific container is right-clicked and a menu opens. Generate SAS is selected from this menu.":::

1. In the **Generate SAS** dialog, specify the **Permissions**, **Start and expiry date/time**, and **Allowed IP addresses**. Then, select **Generate SAS token and URL**
1. In the **Generate SAS** dialog, select **Read** and **List** permissions for containers, or **Read** for individual blobs. Optionally, specify the **Start and expiry date/time** and **Allowed IP addresses**. For more information about the optional parameters, see [best practices when authorizing using SAS](/azure/storage/common/storage-sas-overview#best-practices-when-using-sas). Then select **Generate SAS token and URL**.

:::image type="content" source="storage/generate-sas-token-and-url.png" alt-text="Screen shot of the Generate SAS dialog with information filled in and Generate SAS token and URL selected.":::
:::image type="content" source="storage/generate-sas-token-and-url.png" lightbox="storage/generate-sas-token-and-url.png" alt-text="Screenshot of the Generate SAS dialog with information filled in and Generate SAS token and URL selected.":::

A new section displays at the bottom of the dialog, listing the blob SAS token and the blob SAS URL.

1. Select the icon to the right of the blob SAS URL to copy it. Then, paste it to save or use it as required.
1. Copy the **Blob SAS URL** value and save or use it as required.

:::image type="content" source="storage/copy-sas-token-and-url.png" alt-text="Screenshot of Azure portal with blob SAS URL generated.":::
:::image type="content" source="storage/copy-sas-token-and-url.png" lightbox="storage/copy-sas-token-and-url.png" alt-text="Screenshot of Azure portal with blob SAS URL generated.":::

## Related content

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 43 additions & 8 deletions data-explorer/kusto/management/show-tables-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: .show tables command
description: Learn how to use the `.show tables` command to show a set that contains the specified tables in the database.
ms.reviewer: orspodek
ms.topic: reference
ms.date: 08/11/2024
ms.date: 12/09/2024
---
# .show tables command

Expand Down Expand Up @@ -32,25 +32,60 @@ You must have at least Database User, Database Viewer, or Database Monitor permi

## Returns


|Output parameter |Type |Description
|---|---|---
|TableName | `string` |The name of the table.
|DatabaseName | `string` |The database that the table belongs to.
|Folder | `string` |The table's folder.
|DocString | `string` |A string documenting the table.

## Example
## Examples

### Show all tables in the database

The following example shows information about all tables in the current database.

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA9MrzsgvVyhJTMpJLQYAHfJS8AwAAAA%3D" target="_blank">Run the query</a>
::: moniker-end

```kusto
.show tables
.show tables (T1, ..., Tn)
```

**Output example**
**Output**

|Table Name|Database Name|Folder|DocString|
|---|---|---|---|
|StormEvents|Samples|Storm_Events|US storm events. Data source: https://www.ncdc.noaa.gov/stormevents|
|demo_make_series1|Samples|TimeSeries_and_ML||
|demo_series2|Samples|TimeSeries_and_ML||
|demo_series3|Samples|TimeSeries_and_ML||
|demo_many_series1|Samples|TimeSeries_and_ML||
|ConferenceSessions|Samples|ADX_Conferences||


### Show specific tables

The following example shows information about the specified tables.

:::moniker range="azure-data-explorer"
> [!div class="nextstepaction"]
> <a href="https://dataexplorer.azure.com/clusters/help/databases/Samples?query=H4sIAAAAAAAAA9MrzsgvVyhJTMpJLVbQCMgvKM1JLMnMz3NJLEnUUQguyS%2FKdS1LzSsp1lEIKcosAFLuqflBqelAJcWaAMjc30o9AAAA" target="_blank">Run the query</a>
::: moniker-end

```kusto
.show tables (PopulationData, StormEvents, Trips, GeoRegions)
```

**Output**

|Table Name |Database Name |Folder | DocString
|---|---|---|---
|Table1 |DB1 |Logs |Contains services logs
|Table2 |DB1 | Reporting |
|Table3 |DB1 | | Extended info |
|Table4 |DB2 | Metrics| Contains services performance information
|StormEvents| Samples |Storm_Events| US storm events. Data source: https://www.ncdc.noaa.gov/stormevents|
|PopulationData|Samples| Storm_Events|
|GeoRegions| Samples| NYC Taxi |
|Trips| Samples| NYC Taxi |

6 changes: 3 additions & 3 deletions data-explorer/kusto/query/facet-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ ms.date: 08/11/2024

> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)]

Returns a set of tables, one for each specified column.
Each table specifies the list of values taken by its column.
An additional table can be created by using the `with` clause.
Returns a set of tables, one for each column specified in the facet clause.
Each table contains the list of values taken by its column.
An additional table can be created by using the `with` clause. Facet result tables can't be renamed or referenced by any additional operators.

## Syntax

Expand Down
3 changes: 2 additions & 1 deletion data-explorer/kusto/query/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ StormEvents
| 28|

> [!NOTE]
> KQL is case-sensitive for everything – table names, table column names, operators, functions, and so on.
> KQL is case-sensitive for everything – table names, table column names, operators, functions, and so on.
> Keywords can be used as identifiers by enclosing them in brackets and quotes (`['` and `']` or `["` and `"]`). For example, `['where']`. For more information, see [Identifier naming rules](/kusto/query/schema-entities/entity-names?view=azure-data-explorer&preserve-view=true#identifier-naming-rules)

This query has a single tabular expression statement. The statement begins with a reference to a table called *StormEvents* and contains several operators, [`where`](where-operator.md) and [`count`](count-operator.md), each separated by a pipe. The data rows for the source table are filtered by the value of the *StartTime* column and then filtered by the value of the *State* column. In the last line, the query returns a table with a single column and a single row containing the count of the remaining rows.

Expand Down
Binary file modified data-explorer/media/get-data-storage/add-uri.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading