Skip to content

Commit

Permalink
Add expect_column_distinct_count_to_equal_other_table (#50)
Browse files Browse the repository at this point in the history
  • Loading branch information
clausherther authored Apr 22, 2021
1 parent 9131e47 commit f100088
Show file tree
Hide file tree
Showing 3 changed files with 71 additions and 0 deletions.
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,40 @@ tests:
value_set: ['a','b','c']
```


#### [expect_column_distinct_count_to_equal_other_table](macros/schema_tests/aggregate_functions/expect_column_distinct_count_to_equal_other_table.sql)

Expect the number of distinct column values to be equal to number of distinct values in another model.

*Applies to:* Model, Column, Seed, Source

This can be applied to a model:

```yaml
models: # or seeds:
- name: my_model_1
tests:
- dbt_expectations.expect_column_distinct_count_to_equal_other_table:
column_name: col_1
compare_model: ref("my_model_2")
compare_column_name: col_2
```

or at the column level:

```yaml
models: # or seeds:
- name: my_model_1
columns:
- name: col_1
tests:
- dbt_expectations.expect_column_distinct_count_to_equal_other_table:
compare_model: ref("my_model_2")
compare_column_name: col_2
```

If `compare_model` or `compare_column_name` are no specified, `model` and `column_name` are substituted. So, one could compare distinct counts of two different columns in the same model, or identically named columns in separate models etc.

#### [expect_column_mean_to_be_between](macros/schema_tests/aggregate_functions/expect_column_mean_to_be_between.sql)

Expect the column mean to be between a min_value value and a max_value value (inclusive).
Expand Down
10 changes: 10 additions & 0 deletions integration_tests/models/schema_tests/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ models:
tests:
- dbt_expectations.expect_table_columns_to_match_ordered_list:
column_list: ["date_day", "row_value", "row_value_log"]
- dbt_expectations.expect_column_distinct_count_to_equal_other_table:
column_name: date_day
compare_model: ref("timeseries_data_extended")
compare_column_name: date_day
- dbt_expectations.expect_column_distinct_count_to_equal_other_table:
column_name: date_day

columns:
- name: date_day
tests:
Expand All @@ -38,6 +45,9 @@ models:
column_type_list: [date, "{{ dbt_expectations.type_datetime() | trim }}"]
- dbt_expectations.expect_column_values_to_be_increasing:
sort_column: date_day
- dbt_expectations.expect_column_distinct_count_to_equal_other_table:
compare_model: ref("timeseries_data_extended")


- name: row_value
tests:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{% macro test_expect_column_distinct_count_to_equal_other_table(model,
compare_model,
column_name,
compare_column_name,
row_condition=None,
compare_row_condition=None
) %}
{%- set expression -%}
count(distinct {{ column_name }})
{%- endset -%}
{%- set compare_expression -%}
{%- if compare_column_name -%}
count(distinct {{ compare_column_name }})
{%- else -%}
{{ expression }}
{%- endif -%}
{%- endset -%}
{{ dbt_expectations.test_equal_expression(
model,
expression=expression,
compare_model=compare_model,
compare_expression=compare_expression,
row_condition=row_condition,
compare_row_condition=compare_row_condition,
return_difference=True
) }}
{%- endmacro -%}

0 comments on commit f100088

Please sign in to comment.