Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add labels to DBT models #3675

Merged
merged 3 commits into from
Feb 5, 2025
Merged

Add labels to DBT models #3675

merged 3 commits into from
Feb 5, 2025

Conversation

erikamov
Copy link
Contributor

@erikamov erikamov commented Feb 4, 2025

Description

This PR will add two labels to DBT models helping us to improve billing reports as part of the issue #3541.
Label were based on the current dataset structure.

  • domain:

    • seeds
    • staging
    • intermediate
    • mart
  • dataset:

    • audit
    • benefits (benefits and amplitude datasets)
    • gtfs (gtfs and rt datasets)
    • gtfs_quality
    • gtfs_schedule_latest
    • ntd
    • ntd_annual_reporting
    • ntd_ridership
    • ntd_safety_and_security
    • ntd_validation
    • payments
    • state_geoportal
    • transit_database
    • transit_database_latest

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

Tested locally creating tables on staging.

Screenshot 2025-02-04 at 12 57 43 PM

Screenshot 2025-02-04 at 12 59 19 PM
Screenshot 2025-02-04 at 2 08 28 PM

Post-merge follow-ups

  • No action required
  • Actions required (specified below)

Check if all labels on BigQuery were created after next DAGs' run.

transit_database:
+labels:
domain: intermediate
dataset: transit_database

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Items inside mart: (bellow) were not removed, just reordered alphabetically.

Comment on lines -59 to -60
ad_hoc:
schema: mart_ad_hoc
Copy link
Contributor Author

@erikamov erikamov Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ad-hoc is the only item actually removed, since there is a warning alerting about it when executing poetry run dbt run (see below) and there are no more tables on our codebase under this dataset. Looking through the logs, looks like it was just a testing dataset.

[WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.calitp_warehouse.mart.ad_hoc

@erikamov erikamov force-pushed the add-labels-to-models branch from b7eea38 to 30595dc Compare February 5, 2025 19:18
@erikamov erikamov merged commit 36b9127 into main Feb 5, 2025
4 checks passed
@erikamov erikamov deleted the add-labels-to-models branch February 5, 2025 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants