Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2440] dbt show throws Database Error for models with sql_header required for valid query #7413

Closed
MichelleArk opened this issue Apr 20, 2023 · 2 comments · Fixed by #7568
Assignees
Labels
bug Something isn't working
Milestone

Comments

@MichelleArk
Copy link
Contributor

If a model is configured with a sql_header that is necessary to successfully run the query, dbt show currently fails because the compiled_node.compiled_code does not include the sql_header SQL.

Reproduction case (run against BQ, but not a BQ-specific issue)

-- models/my_model.sql
{% call set_sql_header(config) %}
  CREATE TEMPORARY FUNCTION yes_no_to_boolean(answer STRING)
  RETURNS BOOLEAN AS (
    CASE
    WHEN LOWER(answer) = 'yes' THEN True
    WHEN LOWER(answer) = 'no' THEN False
    ELSE NULL
    END
  );
{%- endcall %}

select yes_no_to_boolean("yes") as column
dbt show --select my_model --project-dir
19:00:05  Found 1 model, 0 tests, 0 snapshots, 0 analyses, 551 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
19:00:05  
19:00:06  Concurrency: 1 threads (target='dev')
19:00:06  
19:00:08  BigQuery adapter: https://console.cloud.google.com/bigquery?project=dbt-test-env&j=bq:US:9802c6ea-f771-4d46-9da3-bf6f521bd1da&page=queryresults
19:00:08  Encountered an error:
Runtime Error
  Database Error in model dummydep (models2/dummydep.sql)
    Function not found: yes_no_to_boolean at [8:8]

Acceptance criteria:
Instead of directly executing compiled_node.compiled_code, template it into a multi-statement query that includes the sql_header (similar approach to the one proposed for #7390)

@MichelleArk MichelleArk added bug Something isn't working Team:Execution labels Apr 20, 2023
@MichelleArk MichelleArk added this to the v1.5.x milestone Apr 20, 2023
@github-actions github-actions bot changed the title dbt show throws Database Error for models with sql_header required for valid query [CT-2440] dbt show throws Database Error for models with sql_header required for valid query Apr 20, 2023
@iknox-fa
Copy link
Contributor

BLG - 4/24 assigned to @aranke for further research

@aranke
Copy link
Member

aranke commented May 8, 2023

Doing some research, the SQL header does indeed run before the query, but is in a different scope.

So, all tests pass in PostgreSQL if I change models_second_model to:

{% call set_sql_header(config) %}
CREATE OR REPLACE FUNCTION increment(i integer) RETURNS integer AS $$
        BEGIN
                RETURN i + 1;
        END;
$$ LANGUAGE plpgsql;
{%- endcall %}
select
    sample_num as col_one,
    sample_bool as col_two,
    increment(41) as answer
from {{ ref('sample_model') }}

It seems like this might only be a problem in BigQuery, so I'm inclined to close this issue for now, please let me know if this is inaccurate.

cc @jtcohen6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants