docs(ingest/dbt): clarify dbt ingestion docs (datahub-project#11312)

sleeperdeep · Sep 9, 2024 · 9b64b11 · 9b64b11
1 parent d0bad4a
commit 9b64b11
Show file tree

Hide file tree

Showing 2 changed files with 14 additions and 12 deletions.
diff --git a/metadata-ingestion/docs/sources/dbt/README.md b/metadata-ingestion/docs/sources/dbt/README.md
@@ -2,17 +2,19 @@ Ingesting metadata from dbt requires either using the **dbt** module or the **db
 
 ### Concept Mapping
 
-| Source Concept  | DataHub Concept                                               | Notes              |
-| --------------- | ------------------------------------------------------------- | ------------------ |
-| `"dbt"`         | [Data Platform](../../metamodel/entities/dataPlatform.md)     |                    |
-| dbt Source      | [Dataset](../../metamodel/entities/dataset.md)                | Subtype `Source`   |
-| dbt Seed        | [Dataset](../../metamodel/entities/dataset.md)                | Subtype `Seed`     |
-| dbt Model       | [Dataset](../../metamodel/entities/dataset.md)                | Subtype `Model`    |
-| dbt Snapshot    | [Dataset](../../metamodel/entities/dataset.md)                | Subtype `Snapshot` |
-| dbt Test        | [Assertion](../../metamodel/entities/assertion.md)            |                    |
-| dbt Test Result | [Assertion Run Result](../../metamodel/entities/assertion.md) |                    |
+| Source Concept | DataHub Concept                                                        | Notes              |
+| -------------- | ---------------------------------------------------------------------- | ------------------ |
+| Source         | [Dataset](../../metamodel/entities/dataset.md)                         | Subtype `Source`   |
+| Seed           | [Dataset](../../metamodel/entities/dataset.md)                         | Subtype `Seed`     |
+| Model          | [Dataset](../../metamodel/entities/dataset.md)                         | Subtype `Model`    |
+| Snapshot       | [Dataset](../../metamodel/entities/dataset.md)                         | Subtype `Snapshot` |
+| Test           | [Assertion](../../metamodel/entities/assertion.md)                     |                    |
+| Test Result    | [Assertion Run Result](../../metamodel/entities/assertion.md)          |                    |
+| Model Runs     | [DataProcessInstance](../../metamodel/entities/dataProcessInstance.md) |                    |
 
 Note:
 
-1. It also generates lineage between the `dbt` nodes (e.g. ephemeral nodes that depend on other dbt sources) as well as lineage between the `dbt` nodes and the underlying (target) platform nodes (e.g. BigQuery Table -> dbt Source, dbt View -> BigQuery View).
-2. We also support automated actions (like add a tag, term or owner) based on properties defined in dbt meta.
+1. You must **run ingestion for both dbt and your data warehouse** (target platform). They can be run in any order.
+2. It generates column lineage between the `dbt` nodes (e.g. when a model/snapshot depends on a dbt source or ephemeral model) as well as lineage between the `dbt` nodes and the underlying target platform nodes (e.g. BigQuery Table -> dbt source, dbt model -> BigQuery table/view).
+3. It automatically generates "sibling" relationships between the dbt nodes and the target / data warehouse nodes. These nodes will show up in the UI with both platform logos.
+4. We also support automated actions (like add a tag, term or owner) based on properties defined in dbt meta.
diff --git a/metadata-ingestion/src/datahub/ingestion/source/aws/aws_common.py b/metadata-ingestion/src/datahub/ingestion/source/aws/aws_common.py
@@ -91,7 +91,7 @@ class AwsConnectionConfig(ConfigModel):
     aws_role: Optional[Union[str, List[Union[str, AwsAssumeRoleConfig]]]] = Field(
         default=None,
         description="AWS roles to assume. If using the string format, the role ARN can be specified directly. "
-        "If using the object format, the role can be specified in the RoleArn field and additional available arguments are documented at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html?highlight=assume_role#STS.Client.assume_role",
+        "If using the object format, the role can be specified in the RoleArn field and additional available arguments are the same as [boto3's STS.Client.assume_role](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts.html?highlight=assume_role#STS.Client.assume_role).",
     )
     aws_profile: Optional[str] = Field(
         default=None,