Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support dots in model names #3246

Closed
1 of 5 tasks
monti-python opened this issue Apr 12, 2021 · 4 comments · Fixed by #3247 or #3460
Closed
1 of 5 tasks

Support dots in model names #3246

monti-python opened this issue Apr 12, 2021 · 4 comments · Fixed by #3247 or #3460
Labels
bug Something isn't working

Comments

@monti-python
Copy link
Contributor

Describe the bug

In our dbt project we are managing multiple schemas. We are namespacing the models with the standard SQL syntax <schema>.<relation>.sql and following this approach. It works quite well for the most part, however when specifying model selectors by FQN (dbt run --model myschema.mytable) the matcher is unable to locate the desired models

Steps To Reproduce

Using a model or seed with dots in their name and trying to run it via FQN selector (dbt run --model model.with.dots) will trigger the issue.

Expected behavior

model.with.dots runs successfully

Screenshots and log output

Log output is:

The selector 'model.with.dots' does not match any nodes and will be ignored

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.19.1
   latest version: 0.19.1

Up to date!

Plugins:
  - bigquery: 0.19.1
  - postgres: 0.19.1
  - redshift: 0.19.1
  - snowflake: 0.19.1

The operating system you're using:

OS Name: Microsoft Windows 10 Enterprise
OS Version: 10.0.18363 N/A Build 18363

The output of python --version:
Python 3.6.13 :: Anaconda, Inc.

Additional context

Related to #1269

@monti-python
Copy link
Contributor Author

Made a PR to solve it #3247

@monti-python
Copy link
Contributor Author

Also fix for the docs site: dbt-labs/dbt-docs#185

@monti-python monti-python changed the title Support dots in model names as namespace separators Support dots in model names Apr 20, 2021
@jtcohen6 jtcohen6 removed the triage label Apr 20, 2021
@jtcohen6
Copy link
Contributor

@monti-python Wow! You've really done your homework. Thanks for opening all these issues and associated PRs, and sorry for the delay in review on my end.

Honestly, I'm pleasantly surprised that models-with-dots-in-them work as well as they do today. It's not a pattern I see often, even though (as you mention) it's something that comes up occasionally on discourse and in broader namespace conversations such as #1269.

I guess the operative questions are:

  • Is this a pattern that dbt should consistently and reliably support? I think yes. To that end, I'll review the PRs you've opened, and may leave comments where certain functionality is now supported for models but not other resource types. I also think we should add tests, to ensure this functionality remains supported in future dbt versions.
  • Is this a pattern we should document, with trade-offs and recommendations? There I'm not so sure. When would dots be preferable to a convention around, say, double underscores?

As I understand it, in order for this to work on Snowflake, you'd need to either:

  • Alias model.with.dots to model_without_dots
  • Quote the identifier name, which opens up a can of quoting/casing worms on Snowflake on particular

@monti-python
Copy link
Contributor Author

monti-python commented Apr 26, 2021

@jtcohen6 thank you for your feedback! I've updated the PR to include some integration tests as well. IMO this feature only makes sense when combined with custom parsing of schema and alias (and potentially db) from the model name, as the dot character is the natural separator for these entities.

At the beginning we were using double underscores as separators, but we came across some relations containing double underscores in their name. Worse yet, there might also be schemas containing double underscores, making it unusable as a separator... Of course there is always the option of a more complex separator (triple underscores for instance), but the dot character felt like the more natural way to tackle the namespacing problem, and it worked!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants