Skip to content

Commit

Permalink
Update cloud cli docs with windows, new instructions (#4172)
Browse files Browse the repository at this point in the history
Added the latest, feel free to change! Changes are still being added by
product team so not ready to fully merge into mega branch yet

merging into #4191

[ ] needs docs review
mirnawong1 authored Oct 13, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
2 parents 2e33572 + 91594f3 commit 0847191
Showing 41 changed files with 525 additions and 266 deletions.
4 changes: 2 additions & 2 deletions website/docs/docs/about-setup.md
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ sidebar_label: "About dbt setup"

dbt compiles and runs your analytics code against your data platform, enabling you and your team to collaborate on a single source of truth for metrics, insights, and business definitions. There are two options for deploying dbt:

**dbt Cloud** runs dbt Core in a hosted (single or multi-tenant) environment with a browser-based interface. The intuitive UI will aid you in setting up the various components. dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring & alerting, and an integrated developer environment (IDE).
**dbt Cloud** runs dbt Core in a hosted (single or multi-tenant) environment with a browser-based interface. The intuitive user interface aids you in setting up the various components. dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring, and alerting. It also offers an integrated development environment (IDE) and allows you to develop and run dbt commands from your local command line (CLI) or code editor.

**dbt Core** is an open-source command line tool that can be installed locally in your environment, and communication with databases is facilitated through adapters.

@@ -19,7 +19,7 @@ To begin configuring dbt now, select the option that is right for you.

<Card
title="dbt Cloud setup"
body="Learn how to connect to a data platform, integrate with secure authentication methods, configure a sync with a git repo, and how to use the IDE."
body="Learn how to connect to a data platform, integrate with secure authentication methods, configure a sync with a git repo, how to use the IDE, and how to install the dbt Cloud CLI."
link="/docs/cloud/about-cloud-setup"
icon="dbt-bit"/>

55 changes: 55 additions & 0 deletions website/docs/docs/cloud/about-cloud-develop-defer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: Using defer in dbt Cloud
id: about-cloud-develop-defer
description: "Learn how to leverage defer to prod when developing with dbt Cloud."
sidebar_label: "Using defer in dbt Cloud"
pagination_next: "docs/cloud/cloud-cli-installation"
---


[Defer](/reference/node-selection/defer) is a powerful feature that allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). This is powered by using a production manifest for comparison, and dbt will resolve the `{{ ref() }}` function with upstream production artifacts.

Both the dbt Cloud IDE and the dbt Cloud CLI allow users to natively defer to production metadata directly in their development workflows, dramatically reducing development time and warehouse spend by preventing unnecessary model builds.

## Required setup

- You must select the **[Production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)** checkbox in the **Environment Settings** page.
- This can be set for one deployment environment per dbt Cloud project.
- You must have a successful job run first.

When using 'defer', it compares artifacts from the most recent successful production job, excluding CI jobs.

### Defer in the dbt Cloud IDE

To enable 'Defer' in the dbt Cloud IDE, toggle the **Defer to production** button on the command bar. Once enabled, dbt Cloud will:

1. Pull down the most recent manifest from the Production environment for comparison
2. Pass the `--defer` flag to the command (for any command that accepts the flag)

For example, if you were to start developing on a new branch with [nothing in your development schema](/reference/node-selection/defer#usage), edit a single model, and run `dbt build -s state:modified` &mdash; only the edited model would run. Any `{{ ref() }}` functions will point to the production location of the referenced models.

<Lightbox src="/img/docs/dbt-cloud/defer-toggle.jpg" width="100%" title="Select the 'Defer to production' toggle on the botom right of the command bar to enable defer in the dbt Cloud IDE."/>

### Defer in dbt Cloud CLI

One key difference between using `--defer` in the dbt Cloud CLI and the dbt Cloud IDE is that `--defer` is *automatically* enabled in the dbt Cloud CLI for all invocations, comparing with production artifacts. You can disable it with the `--no-defer` flag.

The dbt Cloud CLI offers additional flexibility by letting you choose the source environment for deferral artifacts. You can set a `defer-env-id` key in either your `dbt_project.yml` or `dbt_cloud.yml` file. If you do not provide a `defer-env-id` setting, the dbt Cloud CLI will use artifacts from your dbt Cloud environment marked 'Production'.

<File name="dbt_cloud.yml">

```yml
dever-env-id: '123456'
```
</File>
<File name="dbt_project.yml">
```yml
dbt_cloud:
dever-env-id: '123456'
```
</File>
33 changes: 33 additions & 0 deletions website/docs/docs/cloud/about-cloud-develop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
title: About developing in dbt Cloud
id: about-cloud-develop
description: "Learn how to develop your dbt projects using dbt Cloud."
sidebar_label: "About developing in dbt Cloud"
pagination_next: "docs/cloud/cloud-cli-installation"
hide_table_of_contents: true
---

dbt Cloud offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI):

<div className="grid--2-col" >

<Card
title="dbt Cloud CLI"
body="Allows you to develop and run dbt commands from your local command line or code editor against your dbt Cloud development environment."
link="/docs/cloud/cloud-cli-installation"
icon="dbt-bit"/>

<Card
title="dbt Cloud IDE"
body="Develop directly in your browser, making dbt project development efficient by compiling code into SQL and managing project changes seamlessly using an intuitive user interface."
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

</div><br />

The following sections provide detailed instructions on setting up the dbt Cloud CLI and dbt Cloud IDE. To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/quickstarts).


---------
**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).

2 changes: 2 additions & 0 deletions website/docs/docs/cloud/about-cloud-setup.md
Original file line number Diff line number Diff line change
@@ -11,6 +11,8 @@ dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It conta
- Configuring access to [GitHub](/docs/cloud/git/connect-github), [GitLab](/docs/cloud/git/connect-gitlab), or your own [git repo URL](/docs/cloud/git/import-a-project-by-git-url).
- [Managing users and licenses](/docs/cloud/manage-access/seats-and-users)
- [Configuring secure access](/docs/cloud/manage-access/about-user-access)
- Configuring the [dbt Cloud IDE](/docs/cloud/about-cloud-develop)
- Installing and configuring the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation)

These settings are intended for dbt Cloud administrators. If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](/quickstarts).

49 changes: 28 additions & 21 deletions website/docs/docs/cloud/about-cloud/dbt-cloud-features.md
Original file line number Diff line number Diff line change
@@ -6,84 +6,91 @@ description: "Explore dbt Cloud's features and learn why dbt Cloud is the fastes
hide_table_of_contents: true
---

dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI. In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring & alerting, and an integrated development environment (IDE).
dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI.

In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring and alerting, an integrated development environment (IDE), and allows you to develop and run dbt commands from your local command line interface (CLI) or code editor.

dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make it well-suited for data teams of any size &mdash; sign up for your [free 14-day trial](https://www.getdbt.com/signup/)!

<div className="grid--3-col">

<Card
title="dbt Cloud CLI"
body="Use the dbt Cloud CLI to develop, test, run, and version control dbt projects and commands in your dbt Cloud development environment. Collaborate with team members, directly from the command line."
link="/docs/cloud/cloud-cli-installation"
icon="dbt-bit"/>

<Card
title="dbt Cloud IDE"
body="The IDE is the easiest and most efficient way to develop dbt models, allowing you to build, test, run, and version control your dbt projects directly from your browser."
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="pencil-paper"/>
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

<Card
title="Manage environments"
body="Set up and manage separate production and development environments in dbt Cloud to help engineers develop and test code more efficiently, without impacting users or data."
link="/docs/environments-in-dbt"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Schedule and run dbt jobs"
body="Create custom schedules to run your production jobs. Schedule jobs by day of the week, time of day, or a recurring interval. Decrease operating costs by using webhooks to trigger CI jobs and the API to start jobs."
link="/docs/deploy/job-scheduler"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Notifications"
body="Set up and customize job notifications in dbt Cloud to receive email or slack alerts when a job run succeeds, fails, or is cancelled. Notifications alert the right people when something goes wrong instead of waiting for a user to report it."
link="/docs/deploy/job-notifications"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Run visibility"
body="View the history of your runs and the model timing dashboard to help identify where improvements can be made to the scheduled jobs."
link="/docs/deploy/run-visibility"
icon="dbt-bit"/>

<Card
title="Host & share documentation"
body="dbt Cloud hosts and authorizes access to dbt project documentation, allowing you to generate data documentation on a schedule for your project. Invite teammates to dbt Cloud to collaborate and share your project's documentation."
link="/docs/collaborate/build-and-view-your-docs"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Supports GitHub, GitLab, AzureDevOPs"
body="Seamlessly connect your git account to dbt Cloud and provide another layer of security to dbt Cloud. Import new repositories, trigger continuous integration, clone repos using HTTPS, and more!"
link="/docs/cloud/git/connect-github"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Enable Continuous Integration"
body="Configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool in any analyst's belt."
link="/docs/deploy/continuous-integration"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Security"
body="Manage risk with SOC-2 compliance, CI/CD deployment, RBAC, and ELT architecture."
link="https://www.getdbt.com/security/"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="dbt Semantic Layer*"
body="Use the dbt Semantic Layer to define metrics alongside your dbt models and query them from any integrated analytics tool. Get the same answers everywhere, every time."
link="/docs/use-dbt-semantic-layer/dbt-sl"
icon="pencil-paper"/>
icon="dbt-bit"/>

<Card
title="Discovery API*"
body="Enhance your workflow and run ad-hoc queries, browse schema, or query the dbt Semantic Layer. dbt Cloud serves a GraphQL API, which supports arbitrary queries."
link="/docs/dbt-cloud-apis/discovery-api"
icon="pencil-paper"/>
icon="dbt-bit"/>


<Card
title="Run visibility"
body="View the history of your runs and the model timing dashboard to help identify where improvements can be made to the scheduled jobs."
link="/docs/deploy/run-visibility"
icon="pencil-paper"/>

<Card
<Card
title="dbt Explorer*"
body="Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your data pipelines."
link="/docs/collaborate/explore-projects"
icon="pencil-paper"/>
icon="dbt-bit"/>
</div> <br />

*These features are available on [selected plans](https://www.getdbt.com/pricing/).
Loading

0 comments on commit 0847191

Please sign in to comment.