From fe73ab908a9899150f8cf6fd0a31432c04d4b439 Mon Sep 17 00:00:00 2001 From: Shirshanka Das Date: Thu, 8 Sep 2022 00:38:10 -0700 Subject: [PATCH] docs(ingest): improve doc gen, docs for snowflake, looker (#5867) --- .../docs/sources/looker/looker.md | 23 ----- .../docs/sources/looker/looker_pre.md | 62 ++++++++++++ .../docs/sources/looker/lookml.md | 13 --- .../docs/sources/looker/lookml_post.md | 11 +++ .../docs/sources/looker/lookml_pre.md | 84 ++++++++++++++++ .../docs/sources/looker/lookml_recipe.yml | 5 +- .../docs/sources/snowflake/README.md | 29 +++++- ...nowflake-beta.md => snowflake-beta_pre.md} | 0 .../snowflake/snowflake-beta_recipe.yml | 13 +-- ...wflake-usage.md => snowflake-usage_pre.md} | 0 metadata-ingestion/scripts/docgen.py | 96 ++++++++++++++++--- .../src/datahub/ingestion/api/decorators.py | 3 +- .../source/snowflake/snowflake_v2.py | 2 +- 13 files changed, 276 insertions(+), 65 deletions(-) delete mode 100644 metadata-ingestion/docs/sources/looker/looker.md create mode 100644 metadata-ingestion/docs/sources/looker/looker_pre.md delete mode 100644 metadata-ingestion/docs/sources/looker/lookml.md create mode 100644 metadata-ingestion/docs/sources/looker/lookml_post.md create mode 100644 metadata-ingestion/docs/sources/looker/lookml_pre.md rename metadata-ingestion/docs/sources/snowflake/{snowflake-beta.md => snowflake-beta_pre.md} (100%) rename metadata-ingestion/docs/sources/snowflake/{snowflake-usage.md => snowflake-usage_pre.md} (100%) diff --git a/metadata-ingestion/docs/sources/looker/looker.md b/metadata-ingestion/docs/sources/looker/looker.md deleted file mode 100644 index 769d3341dca13f..00000000000000 --- a/metadata-ingestion/docs/sources/looker/looker.md +++ /dev/null @@ -1,23 +0,0 @@ -#### Configuration Notes - -See the [Looker authentication docs](https://docs.looker.com/reference/api-and-integration/api-auth#authentication_with_an_sdk) for the steps to create a client ID and secret. -You need to provide the following permissions for ingestion to work correctly. -``` -access_data -explore -manage_models -see_datagroups -see_lookml -see_lookml_dashboards -see_looks -see_pdts -see_queries -see_schedules -see_sql -see_system_activity -see_user_dashboards -see_users -``` -Here is an example permission set after configuration. -![Looker DataHub Permission Set](./looker_datahub_permission_set.png) - diff --git a/metadata-ingestion/docs/sources/looker/looker_pre.md b/metadata-ingestion/docs/sources/looker/looker_pre.md new file mode 100644 index 00000000000000..762b4d0f5af48e --- /dev/null +++ b/metadata-ingestion/docs/sources/looker/looker_pre.md @@ -0,0 +1,62 @@ +### Pre-Requisites + +#### Set up the right permissions +You need to provide the following permissions for ingestion to work correctly. +``` +access_data +explore +manage_models +see_datagroups +see_lookml +see_lookml_dashboards +see_looks +see_pdts +see_queries +see_schedules +see_sql +see_system_activity +see_user_dashboards +see_users +``` +Here is an example permission set after configuration. +![Looker DataHub Permission Set](./looker_datahub_permission_set.png) + +#### Get an API key + +You need to get an API key for the account with the above privileges to perform ingestion. See the [Looker authentication docs](https://docs.looker.com/reference/api-and-integration/api-auth#authentication_with_an_sdk) for the steps to create a client ID and secret. + + +### Ingestion through UI + +The following video shows you how to get started with ingesting Looker metadata through the UI. + +:::note + +You will need to run `lookml` ingestion through the CLI after you have ingested Looker metadata through the UI. Otherwise you will not be able to see Looker Views and their lineage to your warehouse tables. + +::: + +
+