Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(compliance-forms) update guide for creating form via UI #11936

Merged
merged 20 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion docs-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,25 @@ module.exports = {
type: "doc",
id: "docs/glossary/business-glossary",
},
{
label: "Compliance Forms",
type: "category",
collapsed: true,
items: [
{
type: "doc",
id: "docs/features/feature-guides/compliance-forms/overview",
},
{
type: "doc",
id: "docs/features/feature-guides/compliance-forms/create-a-form",
},
{
type: "doc",
id: "docs/features/feature-guides/compliance-forms/complete-a-form",
},
],
},
{
label: "Data Contract",
type: "doc",
Expand All @@ -164,7 +183,6 @@ module.exports = {
type: "doc",
id: "docs/features/dataset-usage-and-query-history",
},
"docs/features/feature-guides/documentation-forms",
{
label: "Domains",
type: "doc",
Expand Down
8 changes: 4 additions & 4 deletions docs/api/tutorials/forms.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Documentation Forms
# Compliance Forms

## Why Would You Use Documentation Forms?
## Why Would You Use Compliance Forms?

Documentation Forms are a way for end-users to fill out all mandatory attributes associated with a data asset. The form will be dynamically generated based on the definitions provided by administrators and stewards and matching rules.
Compliance Forms are a way for end-users to fill out all mandatory attributes associated with a data asset. The form will be dynamically generated based on the definitions provided by administrators and stewards and matching rules.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compliance Forms are not about "end-users" filling out requirements- end users don't care!
They'd prefer to NOT be filling out requirements.

Instead, Compliance Forms are a way to crowd source critical governance and compliance attributes across your data assets AT SCALE. Our target end user if a Data Governance/Compliance Leader, not the end user. We need to ensure we are always speaking about empowering them, first and foremost. They are the persona we need to click with on this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Learn more about forms in the [Documentation Forms Feature Guide](../../../docs/features/feature-guides/documentation-forms.md).
Learn more about forms in the [Compliance Forms Feature Guide](../../../docs/features/feature-guides/compliance-forms/overview.md).

### Goal Of This Guide
This guide will show you how to
Expand Down
176 changes: 176 additions & 0 deletions docs/features/feature-guides/compliance-forms/complete-a-form.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
---
title: Complete a Form
---

import FeatureAvailability from '@site/src/components/FeatureAvailability';

# Complete a DataHub Compliance Form
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
<FeatureAvailability/>

This guide provides an example of completing a Compliance Form, covering:
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

1. Accessing a Form from an Asset Page or the DataHub Cloud Task Center
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
2. Completing a Form for a single Asset or multiple Assets (DataHub Cloud only)
3. Understanding different Form Question completion states

The example uses the **Governance Initiative 2024**, a Verification Form with 3 Required Questions:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc seems to change format completely, being fully example driven. I would consider either running with that narrative through the entire doc series, with a single Form as the example, or just ditching it and keeping it consistent about how to complete


<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/sample-compliance-form.png"
alt="Sample Compliance Form"/>
</p>

## Accessing a Compliance Form
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

Once you have been assigned to complete a Compliance Form, you will see a **Complete Documentation** or **Complete Verification** option on the right-hand side of an Asset Page:

<p align="center">
<img
width="70%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-from-asset-page.png"
alt="Open Compliance Form from Asset Page"/>
</p>

**DataHub Cloud** users can find all outstanding Compliance Form requests by navigating to the **Task Center**:

<p align="center">
<img
width="80%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-tasks-from-task-center.png"
alt="Open Compliance Form from Task Center"/>
</p>

## Completing a Form for a Single Asset

When filling out a Compliance Form for a single Asset, you'll see a list of Questions tailored to that Asset, with clear labels showing which ones are required. Here's how it works:

- **Question Details:** Each Question specifies if it's required or optional. Required Questions must be completed to submit the Form.
- **Pre-Populated Metadata:** If metadata already exists for a Question, it will appear pre-filled. You can confirm the existing value or make updates as needed.
- **Assignee Contributions:** If another Assignee has already provided a response, their name and the time of submission will be displayed. This gives you visibility into previous input, though you can still update the response.

:::tip
For Verification Forms, after addressing all required Questions, you'll be prompted to provide final sign-off. This ensures all responses are complete and accurate, marking the Form ready for submission.
:::

Once you complete all required responses, the sidebar will update with the status of the Asset:

- **Documented**: All required Questions are completed, Verification is not needed
- **Verified**: All required Questions are completed and Verified

Here's what the **Governance Initiative 2024** Verification Form looks like for `dogs_in_movies` after responding to all Required Questions:

<p align="center">
<img
width="80%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-ready-to-verify.png"
alt="Asset Ready to Verify"/>
</p>

And here's the `dogs_in_movies` sidebar after Verifying all responses:

<p align="center">
<img
width="70%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-asset-verified.png"
alt="Asset is Verified"/>
</p>

### Navigating to the Next Asset

To continue working through the Compliance Forms assigned to you, **use the navigation arrows located in the top-right corner**. These arrows will take you to the next Asset that is still pending Form completion or Verification. Only Assets that require action will appear in this flow, allowing you to focus on the remaining tasks without unnecessary steps.


## Completing a Form Question for Multiple Assets

When you want to provide the same response for a question to multiple assets, you can apply it in bulk by selecting the **By Question** option in the top-right corner. This allows you to navigate through the Form question-by-question and apply the same response to multiple assets.

:::note
Completing Form Questions for multiple Assets is only supported for DataHub Cloud.
:::

### Example: Applying a Response in Bulk
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

Let's look at an example:

1. **Filter Assets**: Filter down to all datasets in the `DEMO_DB` Snowflake schema.
2. **Set a Response**: For the selected question, provide a response. In this case, we'll set the Deletion Date to be `2024-12-31`.
3. **Apply to All Selected Assets**: Use the bulk application feature to apply this response to all filtered assets.

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-tasks-by-question.png"
alt="Apply Response to Multiple Assets"/>
</p>

After setting the response, toggle through each question, providing the necessary responses to combinations of assets.

### Verification for Multiple Assets

For Verification Forms, as you complete responses, you will see the number of assets eligible for Verification in the top-right corner. This makes it easy to track which assets have met the requirements.

<p align="center">
<img
width="70%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-tasks-by-question-ready-to-verify.png"
alt="Multiple Assets ready to Verify"/>
</p>

When you are ready to bulk Verify assets, you will be prompted to confirm that all responses are complete and accurate before proceeding.

<p align="center">
<img
width="70%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-tasks-bulk-verify.png"
alt="Final Bulk Verification"/>
</p>

### Switching Between Views
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

You can easily toggle between the **Complete By Asset** and **Complete By Question** views as needed, ensuring flexibility while completing and verifying the Compliance Forms.

## Understanding Different Form Question Completion States

When completing a Compliance Form, you may encounter various types of questions, each with unique completion states based on existing metadata or prior user responses. This section highlights these scenarios to help you understand how questions can be answered, confirmed, or updated during the process.

**_1. What is the primary use case for this asset?_**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I almost feel like this type of Example based switch would be more part of a Tutorial, as opposed to a Feature Guide


This required Question is asking the Assignee to provide Documentation on how the Asset should be used. Note that there is no text populated in the description, meaning the Asset does not have any documentation at all.

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-incomplete-question.png"
alt="Sample Compliance Form"/>
</p>

**_2. When will this asset be deleted?_**

You may notice that this question has a pre-populated value. When metadata has been populated from a source _outside_ of a Form, users will have the option to update and save the value, or, simply **Confirm** that the value is accurate.

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-confirm-value.png"
alt="Sample Compliance Form"/>
</p>

**_3. Who is the Data Steward of this Asset?_**

Here's an example where a different Form Assignee has already provided an answer through the Compliance Form 3 days ago. All Assignees will still have the option to update the response, but this allows users to see how other Form Assignees have already answered the questions.

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/completion/complete-task-question-answered.png"
alt="Sample Compliance Form"/>
</p>


## FAQ and Troubleshooting

**Why can't I complete a Compliance Form for an Asset?**
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

Forms can be assigned to Asset Owners, specific DataHub Users, or a combination of the two. Please confirm with the Form author that your DataHub user is in-scope for the Form.
159 changes: 159 additions & 0 deletions docs/features/feature-guides/compliance-forms/create-a-form.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
title: Create a Form
---

import FeatureAvailability from '@site/src/components/FeatureAvailability';

# Create a DataHub Compliance Form
<FeatureAvailability/>

This guide will walk you through creating and assigning Compliance Forms, including:

1. Defining your Compliance Form
2. Creating Questions to be completed by assignees
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
3. Selecting the in-scope Assets for the Compliance Form
4. Assigning Forms to specific Users
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
5. Publishing your Form

:::note
Creating and managing Compliance Forms via the UI is only available in DataHub Cloud. If you are deployed with DataHub Core, please see the [Compliance Forms API Guide](../../../api/tutorials/forms.md).
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
:::

## Create and Assign a Compliance Form from the DataHub Cloud UI
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

### Prerequisites
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like how you have these pre-requisites etc. I don't see that on the Complete a Form side. It would be good to spell out the prerequisites to being able to complete a form as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jjoyce0510 the only prereq for completing a form is that it's assigned to you, correct?

Should we also call out that users need Edit privileges?


In order to create or edit a Compliance Form, your DataHub User must have the **Manage Compliance Forms** permission.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

### Step 1: Define your Compliance Form
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

From the navigation bar, head to **Govern** > **Compliance Forms**. Click **+ Create** to start building your Form.

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/list-compliance-forms-before.png"
alt="View of all Compliance Forms"/>
</p>

First up, provide the following details:

1. **Name:** Give your Compliance Form a unique name.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
2. **Description:** Describe the purpose of the Form to help your users understand the purpose of the exercise.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
3. **Type:** Determine the collection type of the Form:
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
- **Verification:** Collect required information and require final verification to complete the Form.
- **Completion:** Collect required information; final verification is not required.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
4. Click **Add Question** to begin setting the requirements for your Form.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/create-compliance-form-add-question.png"
alt="Create a new Compliance Form"/>
</p>

### Step 2: Add Questions to your Form

Next, create Questions you want your users to complete to capture the desired metadata with this Compliance Form. There are five types of Questions that can be created, each of which can be set to be **required to respond**:
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

* **Ownership:** Assign one or more Owners to the Asset, with the option to predefine the set of allowed Owners and/or Ownership Types.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
* _E.g. Who is responsible for ensuring the accuracy of this Dataset?_
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
* **Domain:** Assign a Domain to the Asset, with the option to predefine the set of allowed Domains.
* _E.g. Which Domain does this Dashboard belong to? Sales, Marketing, Finance._
* **Documentation:** Provide Documentation about the Asset and/or Column.
* _E.g. What is the primary use case of this Dataset? What caveats should others be aware of?_
* **Glossary Terms:** Assign one or more Glossary Term to the Asset and/or Column, with the option to predefine the set of allowed Glossary Terms.
* _E.g. What types of personally identifiable information (PII) are included in this Asset? Email, Address, SSN, etc._
* **Structured Properties:** Apply custom properties to an Asset and/or Column.
* _E.g. What date will this Dataset be deprecated and deleted?_

When creating a Question, be sure to give it an easy-to-understand Title, and provide additional context or direction in the Description.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/create-compliance-form-prompt.png"
alt="Create a new Compliance Form prompt"/>
</p>

### Step 3: Assign your Form to relevant Assets
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

Now that you have defined the Questions you want Users to complete, it's now time to select the in-scope Assets for this exercise.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

In the **Assign Assets** section, you can easily target the specific set of Assets that are relevant for this Form with the following steps:
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

1. Add a Condition or Group of Conditions
2. Choose the appropriate filter type, such as:
* Asset Type (Dataset, Chart, etc.)
* Platform (Snowflake, dbt, etc.)
* Domain (Sales, Marketing, Finance, etc.)
* Assigned Owners
* Assigned Glossary Terms
3. Decide between **All**, **Any**, or **None** of the filters should apply
4. Preview the relevant Assets to confirm you have applied the appropriate filters

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/create-compliance-form-assign-assets.png"
alt="Assign assets to a Compliance Form"/>
</p>

### Step 4: Add Recipients to your Form

Now that you have defined the set of Questions to be answered for a set of Assets, it's now time to delegate out to your Users.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

In the **Add Recipients** section, decide who is responsible for completing the Form:

* **Asset Owners:** Any User that is assigned to one of the in-scope Assets will be able to complete the Form. This is useful for larger initiatives when you may not know the full set of Users.
* **Specific Users and/or Groups:** Select a specific set of Users and/or Groups within DataHub. This is useful when Ownership of the Assets may be poorly-defined.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

<p align="center">
<img
width="90%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/create-compliance-form-add-users-or-groups.png"
alt="Assign recipients to a Compliance Form"/>
</p>

### Step 5: Publish your Form

Once you have defined the set of Questions to be completed, the in-scope Assets, and the relevant Recipients, click **Publish** and Users will be able to complete your Form!
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

:::caution
Once you have published a Form, you **cannot** change or add Questions. You can, however, change the set of Assets and/or Assignees for the Form.
:::

Not ready for primetime just yet? No worries! You also have the option to **Save Draft**.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

<p align="center">
<img
width="80%"
src="https://raw.githubusercontent.com/datahub-project/static-assets/refs/heads/main/imgs/compliance_forms/management/create-compliance-form-publish.png"
alt="Publish a Compliance Form"/>
</p>

## FAQ and Troubleshooting

**How does a Compliance Form interact with existing metadata?**
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

If an asset already has existing metadata that is also referenced in a Form Question, users assigned to the Form will have the option to confirm the value and make no changes, overwrite the value, or append additional details.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

**What is the difference between Completion and Verification Forms?**

Both form types are a way to configure a set of optional and/or required questions for DataHub users to complete. When using Verification Forms, users will be presented with a final verification step once all required questions have been completed; you can think of this as a final acknowledgment of the accuracy of information submitted.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

**Can I assign multiple Forms to a single asset?**

You sure can! Please keep in mind that an Asset will only be considered Documented or Verified if all required questions are completed on all assigned Forms.
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

### API Tutorials
maggiehays marked this conversation as resolved.
Show resolved Hide resolved

- [API Guides on Documentation Form](../../../api/tutorials/forms.md)

### Related Features

- [DataHub Properties](../../feature-guides/properties.md)

## Next Steps

Now that you have created a DataHub Compliance Form, you're ready to [Complete a Compliance Form](complete-a-form.md).
Loading
Loading